This job view page is being replaced by Spyglass soon. Check out the new job view.
PRandyzhangx: feat: add registrar liveness probe on Linux
ResultFAILURE
Tests 1 failed / 11 succeeded
Started2021-10-09 02:27
Elapsed53m59s
Revision8baf1dce21f686608bc2bb1c9b3dc625ee9787e1
Refs 1043
job-versionv1.23.0-alpha.3.249+80056f73a614b2
kubetest-version
revisionv1.23.0-alpha.3.249+80056f73a614b2

Test Failures


kubetest Test 43m4s

error during make e2e-test: exit status 2
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 11 Passed Tests

Error lines from build-log.txt

... skipping 221 lines ...

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 11248  100 11248    0     0   174k      0 --:--:-- --:--:-- --:--:--  174k
Downloading https://get.helm.sh/helm-v3.7.0-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
docker pull k8sprow.azurecr.io/azuredisk-csi:v1.8.0-9eaf1e00dfb23da1f5430a51a79908d1683f455a || make container-all push-manifest
Error response from daemon: manifest for k8sprow.azurecr.io/azuredisk-csi:v1.8.0-9eaf1e00dfb23da1f5430a51a79908d1683f455a not found: manifest unknown: manifest tagged by "v1.8.0-9eaf1e00dfb23da1f5430a51a79908d1683f455a" is not found
make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=v1.8.0-9eaf1e00dfb23da1f5430a51a79908d1683f455a -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=9eaf1e00dfb23da1f5430a51a79908d1683f455a -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2021-10-09T02:31:39Z -extldflags "-static""  -mod vendor -o _output/amd64/azurediskplugin.exe ./pkg/azurediskplugin
docker buildx rm container-builder || true
error: no builder "container-builder" found
docker buildx create --use --name=container-builder
container-builder
# enable qemu for arm64 build
# https://github.com/docker/buildx/issues/464#issuecomment-741507760
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
Unable to find image 'tonistiigi/binfmt:latest' locally
... skipping 1752 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 60 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 248 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 108 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 858 lines ...
        - name: csi-resizer
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - '-handle-volume-inuse-error=true'
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
            - name: socket-dir
              mountPath: /csi
... skipping 382 lines ...
Oct  9 02:41:32.562: INFO: >>> kubeConfig: /root/tmp576870111/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename topology
W1009 02:41:33.001355   35048 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Oct  9 02:41:33.001: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled
Oct  9 02:41:33.111: INFO: No PSP annotation exists on dry run pod; assuming PodSecurityPolicy is disabled
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Oct  9 02:41:33.327: INFO: Driver didn't provide topology keys -- skipping
[AfterEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Oct  9 02:41:33.328: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "topology-205" for this suite.


S [SKIPPING] [1.099 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver didn't provide topology keys -- skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:124
------------------------------
... skipping 219 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":19,"completed":1,"skipped":1,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
... skipping 45 lines ...
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Oct  9 02:43:08.231: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volumelimits-2695" for this suite.

•
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits","total":19,"completed":2,"skipped":288,"failed":0}

SSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:43:08.472: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 37 lines ...
Oct  9 02:41:34.531: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com2tcvt] to have phase Bound
Oct  9 02:41:34.641: INFO: PersistentVolumeClaim test.csi.azure.com2tcvt found but phase is Pending instead of Bound.
Oct  9 02:41:36.751: INFO: PersistentVolumeClaim test.csi.azure.com2tcvt found but phase is Pending instead of Bound.
Oct  9 02:41:38.862: INFO: PersistentVolumeClaim test.csi.azure.com2tcvt found and phase=Bound (4.330084456s)
STEP: Creating pod exec-volume-test-dynamicpv-c9w2
STEP: Creating a pod to test exec-volume-test
Oct  9 02:41:39.211: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-c9w2" in namespace "volume-6373" to be "Succeeded or Failed"
Oct  9 02:41:39.324: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 112.790426ms
Oct  9 02:41:41.435: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.223708291s
Oct  9 02:41:43.546: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.33427135s
Oct  9 02:41:45.658: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.446543233s
Oct  9 02:41:47.770: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.558317108s
Oct  9 02:41:49.881: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.669385023s
Oct  9 02:41:51.992: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 12.780422027s
Oct  9 02:41:54.101: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 14.889968668s
Oct  9 02:41:56.213: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 17.001254327s
Oct  9 02:41:58.324: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 19.112572674s
Oct  9 02:42:00.435: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Pending", Reason="", readiness=false. Elapsed: 21.223940704s
Oct  9 02:42:02.546: INFO: Pod "exec-volume-test-dynamicpv-c9w2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.335041243s
STEP: Saw pod success
Oct  9 02:42:02.547: INFO: Pod "exec-volume-test-dynamicpv-c9w2" satisfied condition "Succeeded or Failed"
Oct  9 02:42:02.657: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod exec-volume-test-dynamicpv-c9w2 container exec-container-dynamicpv-c9w2: <nil>
STEP: delete the pod
Oct  9 02:42:02.929: INFO: Waiting for pod exec-volume-test-dynamicpv-c9w2 to disappear
Oct  9 02:42:03.038: INFO: Pod exec-volume-test-dynamicpv-c9w2 no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-c9w2
Oct  9 02:42:03.038: INFO: Deleting pod "exec-volume-test-dynamicpv-c9w2" in namespace "volume-6373"
... skipping 27 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should allow exec of files on the volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:196
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume","total":17,"completed":1,"skipped":91,"failed":0}

SSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:43:15.502: INFO: Driver test.csi.azure.com doesn't support ext3 -- skipping
... skipping 74 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume","total":16,"completed":1,"skipped":22,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:44:00.528: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 20 lines ...
[BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Oct  9 02:44:00.585: INFO: >>> kubeConfig: /root/tmp576870111/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename topology
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Oct  9 02:44:01.128: INFO: Driver didn't provide topology keys -- skipping
[AfterEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Oct  9 02:44:01.129: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "topology-4593" for this suite.


S [SKIPPING] [0.775 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver didn't provide topology keys -- skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:124
------------------------------
... skipping 100 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":13,"completed":1,"skipped":332,"failed":0}

SSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] capacity
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:44:08.861: INFO: Driver test.csi.azure.com doesn't publish storage capacity -- skipping
... skipping 174 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":17,"completed":2,"skipped":146,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand 
  Verify if offline PVC expansion works
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174
... skipping 66 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    Verify if offline PVC expansion works
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand Verify if offline PVC expansion works","total":14,"completed":1,"skipped":83,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:45:27.091: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 216 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":16,"completed":2,"skipped":295,"failed":0}

SSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support file as subpath [LinuxOnly]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:230
... skipping 16 lines ...
Oct  9 02:43:09.385: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comf5kvn] to have phase Bound
Oct  9 02:43:09.494: INFO: PersistentVolumeClaim test.csi.azure.comf5kvn found but phase is Pending instead of Bound.
Oct  9 02:43:11.603: INFO: PersistentVolumeClaim test.csi.azure.comf5kvn found but phase is Pending instead of Bound.
Oct  9 02:43:13.713: INFO: PersistentVolumeClaim test.csi.azure.comf5kvn found and phase=Bound (4.328290829s)
STEP: Creating pod pod-subpath-test-dynamicpv-fjtc
STEP: Creating a pod to test atomic-volume-subpath
Oct  9 02:43:14.045: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-fjtc" in namespace "provisioning-99" to be "Succeeded or Failed"
Oct  9 02:43:14.154: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Pending", Reason="", readiness=false. Elapsed: 109.05704ms
Oct  9 02:43:16.264: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218962507s
Oct  9 02:43:18.375: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Pending", Reason="", readiness=false. Elapsed: 4.329771287s
Oct  9 02:43:20.491: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Pending", Reason="", readiness=false. Elapsed: 6.446127702s
Oct  9 02:43:22.605: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Pending", Reason="", readiness=false. Elapsed: 8.559846829s
Oct  9 02:43:24.716: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Pending", Reason="", readiness=false. Elapsed: 10.670580944s
... skipping 35 lines ...
Oct  9 02:44:40.708: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Running", Reason="", readiness=true. Elapsed: 1m26.662698511s
Oct  9 02:44:42.818: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Running", Reason="", readiness=true. Elapsed: 1m28.772669523s
Oct  9 02:44:44.929: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Running", Reason="", readiness=true. Elapsed: 1m30.883932981s
Oct  9 02:44:47.040: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Running", Reason="", readiness=true. Elapsed: 1m32.994524665s
Oct  9 02:44:49.149: INFO: Pod "pod-subpath-test-dynamicpv-fjtc": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m35.103978703s
STEP: Saw pod success
Oct  9 02:44:49.149: INFO: Pod "pod-subpath-test-dynamicpv-fjtc" satisfied condition "Succeeded or Failed"
Oct  9 02:44:49.258: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-fjtc container test-container-subpath-dynamicpv-fjtc: <nil>
STEP: delete the pod
Oct  9 02:44:49.528: INFO: Waiting for pod pod-subpath-test-dynamicpv-fjtc to disappear
Oct  9 02:44:49.636: INFO: Pod pod-subpath-test-dynamicpv-fjtc no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-fjtc
Oct  9 02:44:49.636: INFO: Deleting pod "pod-subpath-test-dynamicpv-fjtc" in namespace "provisioning-99"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support file as subpath [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:230
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","total":19,"completed":3,"skipped":432,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:45:31.554: INFO: Driver "test.csi.azure.com" does not support volume type "CSIInlineVolume" - skipping
... skipping 121 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":17,"completed":3,"skipped":299,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:47:32.092: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 58 lines ...
Oct  9 02:45:28.517: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com22mhx] to have phase Bound
Oct  9 02:45:28.625: INFO: PersistentVolumeClaim test.csi.azure.com22mhx found but phase is Pending instead of Bound.
Oct  9 02:45:30.733: INFO: PersistentVolumeClaim test.csi.azure.com22mhx found but phase is Pending instead of Bound.
Oct  9 02:45:32.841: INFO: PersistentVolumeClaim test.csi.azure.com22mhx found and phase=Bound (4.324693016s)
STEP: Creating pod pod-subpath-test-dynamicpv-9v5q
STEP: Creating a pod to test subpath
Oct  9 02:45:33.174: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-9v5q" in namespace "provisioning-6500" to be "Succeeded or Failed"
Oct  9 02:45:33.282: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 107.688124ms
Oct  9 02:45:35.391: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.216469343s
Oct  9 02:45:37.505: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.331239705s
Oct  9 02:45:39.615: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.441067462s
Oct  9 02:45:41.724: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.550315253s
Oct  9 02:45:43.849: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.675172853s
... skipping 24 lines ...
Oct  9 02:46:36.596: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.421471594s
Oct  9 02:46:38.705: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m5.530937533s
Oct  9 02:46:40.819: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m7.644784159s
Oct  9 02:46:42.928: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m9.753637331s
Oct  9 02:46:45.037: INFO: Pod "pod-subpath-test-dynamicpv-9v5q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m11.862933659s
STEP: Saw pod success
Oct  9 02:46:45.037: INFO: Pod "pod-subpath-test-dynamicpv-9v5q" satisfied condition "Succeeded or Failed"
Oct  9 02:46:45.146: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-9v5q container test-container-volume-dynamicpv-9v5q: <nil>
STEP: delete the pod
Oct  9 02:46:45.398: INFO: Waiting for pod pod-subpath-test-dynamicpv-9v5q to disappear
Oct  9 02:46:45.505: INFO: Pod pod-subpath-test-dynamicpv-9v5q no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-9v5q
Oct  9 02:46:45.505: INFO: Deleting pod "pod-subpath-test-dynamicpv-9v5q" in namespace "provisioning-6500"
... skipping 29 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directory
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:205
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory","total":14,"completed":2,"skipped":731,"failed":0}

SSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:47:58.045: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 111 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents","total":13,"completed":2,"skipped":703,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support readOnly file specified in the volumeMount [LinuxOnly]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:380
... skipping 16 lines ...
Oct  9 02:48:05.916: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com8d28f] to have phase Bound
Oct  9 02:48:06.024: INFO: PersistentVolumeClaim test.csi.azure.com8d28f found but phase is Pending instead of Bound.
Oct  9 02:48:08.133: INFO: PersistentVolumeClaim test.csi.azure.com8d28f found but phase is Pending instead of Bound.
Oct  9 02:48:10.247: INFO: PersistentVolumeClaim test.csi.azure.com8d28f found and phase=Bound (4.331033122s)
STEP: Creating pod pod-subpath-test-dynamicpv-jxbq
STEP: Creating a pod to test subpath
Oct  9 02:48:10.574: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-jxbq" in namespace "provisioning-5586" to be "Succeeded or Failed"
Oct  9 02:48:10.687: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 112.706129ms
Oct  9 02:48:12.796: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222187556s
Oct  9 02:48:14.907: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 4.33298949s
Oct  9 02:48:17.018: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 6.443457098s
Oct  9 02:48:19.127: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 8.553240051s
Oct  9 02:48:21.238: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 10.663401594s
Oct  9 02:48:23.347: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 12.772672395s
Oct  9 02:48:25.459: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 14.884558665s
Oct  9 02:48:27.568: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 16.993470875s
Oct  9 02:48:29.678: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Pending", Reason="", readiness=false. Elapsed: 19.104029901s
Oct  9 02:48:31.787: INFO: Pod "pod-subpath-test-dynamicpv-jxbq": Phase="Succeeded", Reason="", readiness=false. Elapsed: 21.212577156s
STEP: Saw pod success
Oct  9 02:48:31.787: INFO: Pod "pod-subpath-test-dynamicpv-jxbq" satisfied condition "Succeeded or Failed"
Oct  9 02:48:31.895: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-jxbq container test-container-subpath-dynamicpv-jxbq: <nil>
STEP: delete the pod
Oct  9 02:48:32.145: INFO: Waiting for pod pod-subpath-test-dynamicpv-jxbq to disappear
Oct  9 02:48:32.253: INFO: Pod pod-subpath-test-dynamicpv-jxbq no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-jxbq
Oct  9 02:48:32.253: INFO: Deleting pod "pod-subpath-test-dynamicpv-jxbq" in namespace "provisioning-5586"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly file specified in the volumeMount [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:380
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]","total":13,"completed":3,"skipped":748,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:49:14.283: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 127 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents","total":17,"completed":4,"skipped":461,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes 
  should store data
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
... skipping 82 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","total":14,"completed":3,"skipped":797,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: CSI Ephemeral-volume (default fs)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:50:18.428: INFO: Driver "test.csi.azure.com" does not support volume type "CSIInlineVolume" - skipping
... skipping 110 lines ...
Oct  9 02:49:53.698: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comr6jt5] to have phase Bound
Oct  9 02:49:53.807: INFO: PersistentVolumeClaim test.csi.azure.comr6jt5 found but phase is Pending instead of Bound.
Oct  9 02:49:55.917: INFO: PersistentVolumeClaim test.csi.azure.comr6jt5 found but phase is Pending instead of Bound.
Oct  9 02:49:58.027: INFO: PersistentVolumeClaim test.csi.azure.comr6jt5 found and phase=Bound (4.329700761s)
STEP: Creating pod pod-subpath-test-dynamicpv-x5j7
STEP: Creating a pod to test subpath
Oct  9 02:49:58.359: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-x5j7" in namespace "provisioning-8453" to be "Succeeded or Failed"
Oct  9 02:49:58.473: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 113.860158ms
Oct  9 02:50:00.584: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.224414483s
Oct  9 02:50:02.694: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.334584219s
Oct  9 02:50:04.804: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.444571681s
Oct  9 02:50:06.915: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.555935118s
Oct  9 02:50:09.026: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.666321682s
Oct  9 02:50:11.136: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 12.77704833s
Oct  9 02:50:13.247: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 14.887703171s
Oct  9 02:50:15.359: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 17.000011932s
Oct  9 02:50:17.471: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 19.111797515s
Oct  9 02:50:19.582: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 21.222274058s
Oct  9 02:50:21.693: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.333421709s
STEP: Saw pod success
Oct  9 02:50:21.693: INFO: Pod "pod-subpath-test-dynamicpv-x5j7" satisfied condition "Succeeded or Failed"
Oct  9 02:50:21.803: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-x5j7 container test-container-subpath-dynamicpv-x5j7: <nil>
STEP: delete the pod
Oct  9 02:50:22.072: INFO: Waiting for pod pod-subpath-test-dynamicpv-x5j7 to disappear
Oct  9 02:50:22.181: INFO: Pod pod-subpath-test-dynamicpv-x5j7 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-x5j7
Oct  9 02:50:22.181: INFO: Deleting pod "pod-subpath-test-dynamicpv-x5j7" in namespace "provisioning-8453"
STEP: Creating pod pod-subpath-test-dynamicpv-x5j7
STEP: Creating a pod to test subpath
Oct  9 02:50:22.406: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-x5j7" in namespace "provisioning-8453" to be "Succeeded or Failed"
Oct  9 02:50:22.518: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Pending", Reason="", readiness=false. Elapsed: 112.000372ms
Oct  9 02:50:24.629: INFO: Pod "pod-subpath-test-dynamicpv-x5j7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 2.222394516s
STEP: Saw pod success
Oct  9 02:50:24.629: INFO: Pod "pod-subpath-test-dynamicpv-x5j7" satisfied condition "Succeeded or Failed"
Oct  9 02:50:24.738: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-x5j7 container test-container-subpath-dynamicpv-x5j7: <nil>
STEP: delete the pod
Oct  9 02:50:24.976: INFO: Waiting for pod pod-subpath-test-dynamicpv-x5j7 to disappear
Oct  9 02:50:25.086: INFO: Pod pod-subpath-test-dynamicpv-x5j7 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-x5j7
Oct  9 02:50:25.086: INFO: Deleting pod "pod-subpath-test-dynamicpv-x5j7" in namespace "provisioning-8453"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directories when readOnly specified in the volumeSource
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:395
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","total":17,"completed":5,"skipped":564,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 76 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":17,"completed":6,"skipped":641,"failed":0}

SSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should create read/write inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
... skipping 43 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume","total":17,"completed":7,"skipped":662,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should be able to unmount after the subpath directory is deleted [LinuxOnly]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:445
... skipping 50 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should be able to unmount after the subpath directory is deleted [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:445
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","total":17,"completed":8,"skipped":784,"failed":0}

SSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning 
  should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
... skipping 100 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":17,"completed":9,"skipped":805,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should support two pods which share the same volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183
... skipping 15 lines ...
Oct  9 02:41:33.881: INFO: Deleting pod "inline-volume-xcp47" in namespace "ephemeral-5454"
Oct  9 02:41:33.993: INFO: Wait up to 5m0s for pod "inline-volume-xcp47" to be fully deleted
Oct  9 02:41:34.102: INFO: Creating resource for dynamic PV
Oct  9 02:41:34.102: INFO: Using claimSize:5Gi, test suite supported size:{ }, driver(test.csi.azure.com) supported size:{ } 
STEP: creating a StorageClass ephemeral-5454-e2e-sczs94p
STEP: checking the requested inline volume exists in the pod running on node {Name: Selector:map[] Affinity:nil}
Oct  9 02:56:59.100: FAIL: waiting for second pod with inline volume
Unexpected error:
    <*errors.errorString | 0xc0002c2240>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred

... skipping 54 lines ...
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:41:58 +0000 UTC - event for inline-volume-tester2-jmvp4: {default-scheduler } FailedScheduling: 0/3 nodes are available: 3 persistentvolumeclaim "inline-volume-tester2-jmvp4-my-volume-0" not found.
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:41:58 +0000 UTC - event for inline-volume-tester2-jmvp4-my-volume-0: {persistentvolume-controller } WaitForPodScheduled: waiting for pod inline-volume-tester2-jmvp4 to be scheduled
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:42:00 +0000 UTC - event for inline-volume-tester2-jmvp4-my-volume-0: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:42:00 +0000 UTC - event for inline-volume-tester2-jmvp4-my-volume-0: {test.csi.azure.com_k8s-agentpool1-31084797-vmss000001_c8870ba8-b183-42b6-a4fc-93b960f3e7f2 } Provisioning: External provisioner is provisioning volume for claim "ephemeral-5454/inline-volume-tester2-jmvp4-my-volume-0"
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:42:02 +0000 UTC - event for inline-volume-tester2-jmvp4-my-volume-0: {test.csi.azure.com_k8s-agentpool1-31084797-vmss000001_c8870ba8-b183-42b6-a4fc-93b960f3e7f2 } ProvisioningSucceeded: Successfully provisioned volume pvc-aaec181d-b96b-4280-9361-eab52a2a07d5
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:42:03 +0000 UTC - event for inline-volume-tester2-jmvp4: {default-scheduler } Scheduled: Successfully assigned ephemeral-5454/inline-volume-tester2-jmvp4 to k8s-agentpool1-31084797-vmss000001
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:42:35 +0000 UTC - event for inline-volume-tester2-jmvp4: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" : rpc error: code = Unknown desc = Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 2) nor in diskMap(map[])
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:42:40 +0000 UTC - event for inline-volume-tester2-jmvp4: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" : rpc error: code = Unknown desc = Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 32) nor in diskMap(map[])
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:43:43 +0000 UTC - event for inline-volume-tester2-jmvp4: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" : rpc error: code = Unknown desc = Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
Oct  9 02:58:44.138: INFO: At 2021-10-09 02:44:06 +0000 UTC - event for inline-volume-tester2-jmvp4: {kubelet k8s-agentpool1-31084797-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[my-volume-0], unattached volumes=[my-volume-0 kube-api-access-h74ch]: timed out waiting for the condition
... skipping 112 lines ...
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183

    Oct  9 02:56:59.100: waiting for second pod with inline volume
    Unexpected error:
        <*errors.errorString | 0xc0002c2240>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:201
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume","total":14,"completed":0,"skipped":198,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:58:47.229: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 215 lines ...
Oct  9 02:58:34.798: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comcv8k8] to have phase Bound
Oct  9 02:58:34.907: INFO: PersistentVolumeClaim test.csi.azure.comcv8k8 found but phase is Pending instead of Bound.
Oct  9 02:58:37.018: INFO: PersistentVolumeClaim test.csi.azure.comcv8k8 found but phase is Pending instead of Bound.
Oct  9 02:58:39.128: INFO: PersistentVolumeClaim test.csi.azure.comcv8k8 found and phase=Bound (4.329781776s)
STEP: Creating pod pod-subpath-test-dynamicpv-swd6
STEP: Creating a pod to test subpath
Oct  9 02:58:39.459: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-swd6" in namespace "provisioning-4773" to be "Succeeded or Failed"
Oct  9 02:58:39.568: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 109.524309ms
Oct  9 02:58:41.680: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221530477s
Oct  9 02:58:43.791: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.33274614s
Oct  9 02:58:45.902: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.443762012s
Oct  9 02:58:48.013: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.554258231s
Oct  9 02:58:50.124: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.66531356s
Oct  9 02:58:52.237: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 12.778448868s
Oct  9 02:58:54.347: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 14.888328624s
Oct  9 02:58:56.458: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 16.998847171s
Oct  9 02:58:58.568: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 19.109612674s
Oct  9 02:59:00.678: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Pending", Reason="", readiness=false. Elapsed: 21.219239315s
Oct  9 02:59:02.788: INFO: Pod "pod-subpath-test-dynamicpv-swd6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.329412579s
STEP: Saw pod success
Oct  9 02:59:02.788: INFO: Pod "pod-subpath-test-dynamicpv-swd6" satisfied condition "Succeeded or Failed"
Oct  9 02:59:02.898: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-swd6 container test-container-subpath-dynamicpv-swd6: <nil>
STEP: delete the pod
Oct  9 02:59:03.128: INFO: Waiting for pod pod-subpath-test-dynamicpv-swd6 to disappear
Oct  9 02:59:03.237: INFO: Pod pod-subpath-test-dynamicpv-swd6 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-swd6
Oct  9 02:59:03.237: INFO: Deleting pod "pod-subpath-test-dynamicpv-swd6" in namespace "provisioning-4773"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing single file [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:219
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","total":17,"completed":10,"skipped":841,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 02:59:45.183: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 37 lines ...
Oct  9 02:59:45.986: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comccslg] to have phase Bound
Oct  9 02:59:46.095: INFO: PersistentVolumeClaim test.csi.azure.comccslg found but phase is Pending instead of Bound.
Oct  9 02:59:48.206: INFO: PersistentVolumeClaim test.csi.azure.comccslg found but phase is Pending instead of Bound.
Oct  9 02:59:50.316: INFO: PersistentVolumeClaim test.csi.azure.comccslg found and phase=Bound (4.32969636s)
STEP: Creating pod pod-subpath-test-dynamicpv-65pf
STEP: Creating a pod to test subpath
Oct  9 02:59:50.648: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-65pf" in namespace "provisioning-3343" to be "Succeeded or Failed"
Oct  9 02:59:50.760: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 111.347369ms
Oct  9 02:59:52.871: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222687592s
Oct  9 02:59:54.983: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.335026199s
Oct  9 02:59:57.095: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 6.446233728s
Oct  9 02:59:59.206: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 8.557925818s
Oct  9 03:00:01.317: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 10.668926538s
Oct  9 03:00:03.428: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 12.779324425s
Oct  9 03:00:05.539: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 14.890739392s
Oct  9 03:00:07.649: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 17.000728626s
Oct  9 03:00:09.760: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 19.111611386s
Oct  9 03:00:11.871: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Pending", Reason="", readiness=false. Elapsed: 21.222576438s
Oct  9 03:00:13.982: INFO: Pod "pod-subpath-test-dynamicpv-65pf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.333322101s
STEP: Saw pod success
Oct  9 03:00:13.982: INFO: Pod "pod-subpath-test-dynamicpv-65pf" satisfied condition "Succeeded or Failed"
Oct  9 03:00:14.092: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-65pf container test-container-volume-dynamicpv-65pf: <nil>
STEP: delete the pod
Oct  9 03:00:14.324: INFO: Waiting for pod pod-subpath-test-dynamicpv-65pf to disappear
Oct  9 03:00:14.433: INFO: Pod pod-subpath-test-dynamicpv-65pf no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-65pf
Oct  9 03:00:14.433: INFO: Deleting pod "pod-subpath-test-dynamicpv-65pf" in namespace "provisioning-3343"
... skipping 22 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support non-existent path
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:194
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path","total":17,"completed":11,"skipped":900,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
Oct  9 03:00:51.255: INFO: Running AfterSuite actions on all nodes
Oct  9 03:00:51.255: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Oct  9 03:00:51.255: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 22 lines ...
Oct  9 02:45:32.616: INFO: Deleting pod "inline-volume-mbht7" in namespace "ephemeral-7339"
Oct  9 02:45:32.728: INFO: Wait up to 5m0s for pod "inline-volume-mbht7" to be fully deleted
Oct  9 02:45:32.838: INFO: Creating resource for dynamic PV
Oct  9 02:45:32.838: INFO: Using claimSize:5Gi, test suite supported size:{ }, driver(test.csi.azure.com) supported size:{ } 
STEP: creating a StorageClass ephemeral-7339-e2e-sctfplw
STEP: checking the requested inline volume exists in the pod running on node {Name: Selector:map[] Affinity:nil}
Oct  9 03:00:33.388: FAIL: waiting for pod with inline volume
Unexpected error:
    <*errors.errorString | 0xc00024e250>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred

... skipping 7 lines ...
k8s.io/kubernetes/test/e2e.TestE2E(0xc0007eea80)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b
testing.tRunner(0xc0007eea80, 0x72d4380)
	/usr/local/go/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x2b3
Oct  9 03:00:33.501: INFO: Error getting logs for pod inline-volume-tester-hn872: the server rejected our request for an unknown reason (get pods inline-volume-tester-hn872)
Oct  9 03:00:33.935: INFO: Deleting pod "inline-volume-tester-hn872" in namespace "ephemeral-7339"
Oct  9 03:00:34.046: INFO: Wait up to 5m0s for pod "inline-volume-tester-hn872" to be fully deleted
Oct  9 03:01:14.267: INFO: Wait up to 5m0s for pod PV pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a to be fully deleted
Oct  9 03:01:14.267: INFO: Waiting up to 5m0s for PersistentVolume pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a to get deleted
Oct  9 03:01:14.376: INFO: PersistentVolume pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a found and phase=Released (108.850537ms)
Oct  9 03:01:19.488: INFO: PersistentVolume pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a found and phase=Released (5.220606737s)
... skipping 13 lines ...
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:45:33 +0000 UTC - event for inline-volume-tester-hn872: {default-scheduler } FailedScheduling: 0/3 nodes are available: 3 persistentvolumeclaim "inline-volume-tester-hn872-my-volume-0" not found.
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:45:33 +0000 UTC - event for inline-volume-tester-hn872-my-volume-0: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:45:33 +0000 UTC - event for inline-volume-tester-hn872-my-volume-0: {test.csi.azure.com_k8s-agentpool1-31084797-vmss000001_c8870ba8-b183-42b6-a4fc-93b960f3e7f2 } Provisioning: External provisioner is provisioning volume for claim "ephemeral-7339/inline-volume-tester-hn872-my-volume-0"
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:45:34 +0000 UTC - event for inline-volume-tester-hn872: {default-scheduler } FailedScheduling: 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:45:35 +0000 UTC - event for inline-volume-tester-hn872-my-volume-0: {test.csi.azure.com_k8s-agentpool1-31084797-vmss000001_c8870ba8-b183-42b6-a4fc-93b960f3e7f2 } ProvisioningSucceeded: Successfully provisioned volume pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:45:36 +0000 UTC - event for inline-volume-tester-hn872: {default-scheduler } Scheduled: Successfully assigned ephemeral-7339/inline-volume-tester-hn872 to k8s-agentpool1-31084797-vmss000001
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:45:36 +0000 UTC - event for inline-volume-tester-hn872: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" : rpc error: code = Unknown desc = Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
Oct  9 03:01:50.375: INFO: At 2021-10-09 02:47:39 +0000 UTC - event for inline-volume-tester-hn872: {kubelet k8s-agentpool1-31084797-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[my-volume-0], unattached volumes=[my-volume-0 kube-api-access-n42bw]: timed out waiting for the condition
... skipping 102 lines ...
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183

    Oct  9 03:00:33.389: waiting for pod with inline volume
    Unexpected error:
        <*errors.errorString | 0xc00024e250>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:321
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume","total":19,"completed":3,"skipped":532,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 03:01:53.263: INFO: Driver test.csi.azure.com doesn't support ext3 -- skipping
... skipping 75 lines ...
Oct  9 02:45:32.487: INFO: Deleting pod "inline-volume-k8z45" in namespace "ephemeral-526"
Oct  9 02:45:32.598: INFO: Wait up to 5m0s for pod "inline-volume-k8z45" to be fully deleted
Oct  9 02:45:32.709: INFO: Creating resource for dynamic PV
Oct  9 02:45:32.709: INFO: Using claimSize:5Gi, test suite supported size:{ }, driver(test.csi.azure.com) supported size:{ } 
STEP: creating a StorageClass ephemeral-526-e2e-scm29gp
STEP: checking the requested inline volume exists in the pod running on node {Name: Selector:map[] Affinity:nil}
Oct  9 03:00:33.256: FAIL: waiting for pod with inline volume
Unexpected error:
    <*errors.errorString | 0xc0002c4250>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred

... skipping 7 lines ...
k8s.io/kubernetes/test/e2e.TestE2E(0xc0003bdb00)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:136 +0x2b
testing.tRunner(0xc0003bdb00, 0x72d4380)
	/usr/local/go/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x2b3
Oct  9 03:00:33.406: INFO: Error getting logs for pod inline-volume-tester-qwk8d: the server rejected our request for an unknown reason (get pods inline-volume-tester-qwk8d)
Oct  9 03:00:33.837: INFO: Deleting pod "inline-volume-tester-qwk8d" in namespace "ephemeral-526"
Oct  9 03:00:33.948: INFO: Wait up to 5m0s for pod "inline-volume-tester-qwk8d" to be fully deleted
Oct  9 03:01:20.171: INFO: Wait up to 5m0s for pod PV pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d to be fully deleted
Oct  9 03:01:20.171: INFO: Waiting up to 5m0s for PersistentVolume pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d to get deleted
Oct  9 03:01:20.279: INFO: PersistentVolume pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d found and phase=Released (107.862007ms)
Oct  9 03:01:25.388: INFO: PersistentVolume pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d found and phase=Released (5.216867079s)
... skipping 13 lines ...
Oct  9 03:01:56.270: INFO: At 2021-10-09 02:45:32 +0000 UTC - event for inline-volume-tester-qwk8d: {default-scheduler } FailedScheduling: 0/3 nodes are available: 3 persistentvolumeclaim "inline-volume-tester-qwk8d-my-volume-0" not found.
Oct  9 03:01:56.270: INFO: At 2021-10-09 02:45:32 +0000 UTC - event for inline-volume-tester-qwk8d-my-volume-0: {persistentvolume-controller } WaitForPodScheduled: waiting for pod inline-volume-tester-qwk8d to be scheduled
Oct  9 03:01:56.270: INFO: At 2021-10-09 02:45:34 +0000 UTC - event for inline-volume-tester-qwk8d-my-volume-0: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Oct  9 03:01:56.270: INFO: At 2021-10-09 02:45:34 +0000 UTC - event for inline-volume-tester-qwk8d-my-volume-0: {test.csi.azure.com_k8s-agentpool1-31084797-vmss000001_c8870ba8-b183-42b6-a4fc-93b960f3e7f2 } Provisioning: External provisioner is provisioning volume for claim "ephemeral-526/inline-volume-tester-qwk8d-my-volume-0"
Oct  9 03:01:56.270: INFO: At 2021-10-09 02:45:36 +0000 UTC - event for inline-volume-tester-qwk8d-my-volume-0: {test.csi.azure.com_k8s-agentpool1-31084797-vmss000001_c8870ba8-b183-42b6-a4fc-93b960f3e7f2 } ProvisioningSucceeded: Successfully provisioned volume pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d
Oct  9 03:01:56.270: INFO: At 2021-10-09 02:45:37 +0000 UTC - event for inline-volume-tester-qwk8d: {default-scheduler } Scheduled: Successfully assigned ephemeral-526/inline-volume-tester-qwk8d to k8s-agentpool1-31084797-vmss000001
Oct  9 03:01:56.271: INFO: At 2021-10-09 02:45:37 +0000 UTC - event for inline-volume-tester-qwk8d: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" : rpc error: code = Unknown desc = Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
Oct  9 03:01:56.271: INFO: At 2021-10-09 02:47:40 +0000 UTC - event for inline-volume-tester-qwk8d: {kubelet k8s-agentpool1-31084797-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[my-volume-0], unattached volumes=[kube-api-access-xfvzc my-volume-0]: timed out waiting for the condition
Oct  9 03:01:56.271: INFO: At 2021-10-09 02:49:54 +0000 UTC - event for inline-volume-tester-qwk8d: {kubelet k8s-agentpool1-31084797-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[my-volume-0], unattached volumes=[my-volume-0 kube-api-access-xfvzc]: timed out waiting for the condition
Oct  9 03:01:56.271: INFO: At 2021-10-09 02:51:55 +0000 UTC - event for inline-volume-tester-qwk8d: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" : rpc error: code = Unknown desc = Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d) in current disk list(len: 32) nor in diskMap(map[])
Oct  9 03:01:56.378: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Oct  9 03:01:56.378: INFO: 
Oct  9 03:01:56.596: INFO: 
Logging node info for node k8s-agentpool1-31084797-vmss000000
Oct  9 03:01:56.704: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-31084797-vmss000000    b65d4125-fc27-4bd0-a024-bbfce324d532 8589 0 2021-10-09 02:30:02 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS4_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-enl5pco2 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-31084797-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS4_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:westeurope topology.kubernetes.io/zone:0 topology.test.csi.azure.com/zone:] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-31084797-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-10-09 02:30:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-10-09 02:30:16 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-10-09 02:30:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {cloud-node-manager Update v1 2021-10-09 02:31:01 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {cloud-controller-manager Update v1 2021-10-09 02:31:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2021-10-09 03:01:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.test.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{29449355264 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{28662923264 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-10-09 02:31:21 +0000 UTC,LastTransitionTime:2021-10-09 02:31:21 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-10-09 03:01:06 +0000 UTC,LastTransitionTime:2021-10-09 02:29:57 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-10-09 03:01:06 +0000 UTC,LastTransitionTime:2021-10-09 02:29:57 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-10-09 03:01:06 +0000 UTC,LastTransitionTime:2021-10-09 02:29:57 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-10-09 03:01:06 +0000 UTC,LastTransitionTime:2021-10-09 02:30:18 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-31084797-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:af18ebc71da74718871b239a303cf70e,SystemUUID:928f3b14-9467-064c-ae00-622732fddbf5,BootID:37554740-d189-4ef0-951b-06d497928d8a,KernelVersion:5.4.0-1059-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.7+azure,KubeletVersion:v1.22.2,KubeProxyVersion:v1.22.2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:56da9b401b7f72bf8cddba52fa044cdf19bd32374892a667a497e3625d34e698 k8sprow.azurecr.io/azuredisk-csi:v1.8.0-9eaf1e00dfb23da1f5430a51a79908d1683f455a],SizeBytes:237774703,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi@sha256:91972069fc98aae86e3297b054cb0a347b256d0a81799337e1ec0c196c341065 mcr.microsoft.com/k8s/csi/azurefile-csi:v1.6.0],SizeBytes:223047280,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi@sha256:d1e335c6d52dc8b0f7db7650879e1673fda5c561761a52f9ba72a216d9ca0721 mcr.microsoft.com/k8s/csi/azuredisk-csi:v1.7.0],SizeBytes:212174459,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver@sha256:6c37c0efdaf49d24dafcced601fb78750205670e4e41842ab3a71935ed23b753 mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.2],SizeBytes:128401987,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:758db666ac7028534dba72e7e9bb1e57bb81b8196f976f7a5cc351ef8b3529e1 k8s.gcr.io/e2e-test-images/agnhost:2.32],SizeBytes:125910179,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager@sha256:76fa95d463bf8950324d68f9f53063bb73628adcc862708f4534d6f44546df63 mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.2],SizeBytes:121930716,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:b1ec3f3090fc799799bee98619a33300d8ee9095219adc4bb688084718810d23 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.2],SizeBytes:103625747,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:29ba45b98c8035adea245f124d08d54dffbd27068940edd879cfd86022db20f9 mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.0],SizeBytes:99406403,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:682d3a1a48abbe03df05e1084923ab668a6651fd80327f9694b0cf302cc49f90 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:63501445,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:0457773d5575596d4f898913b45293f77dea0d955bc2002a4dbf7ea827cfe4f0 mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.2],SizeBytes:52626428,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ea0065331ce28081b4a048bd18530bbe4fe675dacb3c9515a2022ef10abf95ff mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.1],SizeBytes:50901571,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:c24707c5f947d9e384d0f111469ca3266a01c48de6557f306e718c23d699cadf mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.1],SizeBytes:50295363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:1c23b17c78ffbc90c068ee088b77456284b9649952fa127b0f86a0ccff575539 mcr.microsoft.com/oss/kubernetes/coredns:1.7.0],SizeBytes:45233936,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:753e3ca79dc833feed24ee4107ff4f06a96abbd6cefba8d41d7dec6f4d3e75f0 mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.7.1],SizeBytes:40051560,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:f1205ba7ae0b0783bb19442064f4d7d664f3dc0c9b0b5b33358e5eef5aedc554 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.3.0],SizeBytes:18746317,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:3770688d4efa33f8f284cc2ef58d33efa4ff606147e0a8de20e67458e13fedc2 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.4.0],SizeBytes:17183286,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b k8s.gcr.io/e2e-test-images/nginx:1.14-1],SizeBytes:16028126,},ContainerImage{Names:[busybox@sha256:f7ca5a32c10d51aeda3b4d01c61c6061f497893d7f6628b92f822f7117182a57 busybox:latest],SizeBytes:1235724,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 k8s.gcr.io/e2e-test-images/busybox:1.29-1],SizeBytes:1154361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Oct  9 03:01:56.705: INFO: 
... skipping 94 lines ...
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149

    Oct  9 03:00:33.256: waiting for pod with inline volume
    Unexpected error:
        <*errors.errorString | 0xc0002c4250>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:321
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume","total":16,"completed":2,"skipped":305,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand 
  Verify if offline PVC expansion works
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174
... skipping 64 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    Verify if offline PVC expansion works
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand Verify if offline PVC expansion works","total":14,"completed":1,"skipped":827,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
Oct  9 03:02:15.000: INFO: Running AfterSuite actions on all nodes
Oct  9 03:02:15.000: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func17.2
Oct  9 03:02:15.000: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2
... skipping 25 lines ...
Oct  9 03:01:59.918: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.combxkkq] to have phase Bound
Oct  9 03:02:00.026: INFO: PersistentVolumeClaim test.csi.azure.combxkkq found but phase is Pending instead of Bound.
Oct  9 03:02:02.136: INFO: PersistentVolumeClaim test.csi.azure.combxkkq found but phase is Pending instead of Bound.
Oct  9 03:02:04.245: INFO: PersistentVolumeClaim test.csi.azure.combxkkq found and phase=Bound (4.326658966s)
STEP: Expanding non-expandable pvc
Oct  9 03:02:04.461: INFO: currentPvcSize {{5368709120 0} {<nil>} 5Gi BinarySI}, newSize {{6442450944 0} {<nil>}  BinarySI}
Oct  9 03:02:04.679: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:06.898: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:08.898: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:10.897: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:12.897: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:14.897: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:16.899: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:18.897: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:20.897: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:22.896: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:24.900: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:26.897: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:28.898: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:30.899: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:32.896: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:34.896: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:02:35.114: INFO: Error updating pvc test.csi.azure.combxkkq: persistentvolumeclaims "test.csi.azure.combxkkq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
STEP: Deleting pvc
Oct  9 03:02:35.114: INFO: Deleting PersistentVolumeClaim "test.csi.azure.combxkkq"
Oct  9 03:02:35.228: INFO: Waiting up to 5m0s for PersistentVolume pvc-4df82415-befc-4e55-bab9-b5ba96c29e46 to get deleted
Oct  9 03:02:35.335: INFO: PersistentVolume pvc-4df82415-befc-4e55-bab9-b5ba96c29e46 found and phase=Released (107.558201ms)
Oct  9 03:02:40.444: INFO: PersistentVolume pvc-4df82415-befc-4e55-bab9-b5ba96c29e46 was removed
STEP: Deleting sc
... skipping 8 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not allow expansion of pvcs without AllowVolumeExpansion property
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:157
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property","total":16,"completed":3,"skipped":400,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume"]}

SSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 03:02:40.998: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 215 lines ...
Oct  9 03:01:54.108: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comlbz57] to have phase Bound
Oct  9 03:01:54.217: INFO: PersistentVolumeClaim test.csi.azure.comlbz57 found but phase is Pending instead of Bound.
Oct  9 03:01:56.327: INFO: PersistentVolumeClaim test.csi.azure.comlbz57 found but phase is Pending instead of Bound.
Oct  9 03:01:58.437: INFO: PersistentVolumeClaim test.csi.azure.comlbz57 found and phase=Bound (4.329155113s)
STEP: Creating pod pod-subpath-test-dynamicpv-vzxz
STEP: Creating a pod to test subpath
Oct  9 03:01:58.767: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-vzxz" in namespace "provisioning-9114" to be "Succeeded or Failed"
Oct  9 03:01:58.877: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 109.868631ms
Oct  9 03:02:00.987: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220543479s
Oct  9 03:02:03.099: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332229496s
Oct  9 03:02:05.210: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 6.44315287s
Oct  9 03:02:07.321: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 8.554624289s
Oct  9 03:02:09.432: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 10.665773472s
... skipping 9 lines ...
Oct  9 03:02:30.547: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 31.780759646s
Oct  9 03:02:32.658: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 33.891491336s
Oct  9 03:02:34.769: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 36.002064877s
Oct  9 03:02:36.880: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Pending", Reason="", readiness=false. Elapsed: 38.113692336s
Oct  9 03:02:38.991: INFO: Pod "pod-subpath-test-dynamicpv-vzxz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.224346291s
STEP: Saw pod success
Oct  9 03:02:38.991: INFO: Pod "pod-subpath-test-dynamicpv-vzxz" satisfied condition "Succeeded or Failed"
Oct  9 03:02:39.101: INFO: Trying to get logs from node k8s-agentpool1-31084797-vmss000000 pod pod-subpath-test-dynamicpv-vzxz container test-container-subpath-dynamicpv-vzxz: <nil>
STEP: delete the pod
Oct  9 03:02:39.335: INFO: Waiting for pod pod-subpath-test-dynamicpv-vzxz to disappear
Oct  9 03:02:39.445: INFO: Pod pod-subpath-test-dynamicpv-vzxz no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-vzxz
Oct  9 03:02:39.445: INFO: Deleting pod "pod-subpath-test-dynamicpv-vzxz" in namespace "provisioning-9114"
... skipping 23 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly directory specified in the volumeMount
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","total":19,"completed":4,"skipped":618,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should support multiple inline ephemeral volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:221
... skipping 64 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support multiple inline ephemeral volumes
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:221
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":19,"completed":5,"skipped":670,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits 
  should support volume limits [Serial]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumelimits.go:123
... skipping 76 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support volume limits [Serial]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumelimits.go:123
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should support volume limits [Serial]","total":9,"completed":1,"skipped":330,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 03:07:28.663: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 37 lines ...
Oct  9 03:07:29.467: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comlznpd] to have phase Bound
Oct  9 03:07:29.575: INFO: PersistentVolumeClaim test.csi.azure.comlznpd found but phase is Pending instead of Bound.
Oct  9 03:07:31.685: INFO: PersistentVolumeClaim test.csi.azure.comlznpd found but phase is Pending instead of Bound.
Oct  9 03:07:33.793: INFO: PersistentVolumeClaim test.csi.azure.comlznpd found and phase=Bound (4.326614919s)
STEP: Expanding non-expandable pvc
Oct  9 03:07:34.038: INFO: currentPvcSize {{5368709120 0} {<nil>} 5Gi BinarySI}, newSize {{6442450944 0} {<nil>}  BinarySI}
Oct  9 03:07:34.256: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:36.477: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:38.476: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:40.478: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:42.477: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:44.477: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:46.478: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:48.476: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:50.478: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:52.479: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:54.476: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:56.478: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:07:58.476: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:08:00.479: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:08:02.478: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:08:04.481: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Oct  9 03:08:04.701: INFO: Error updating pvc test.csi.azure.comlznpd: persistentvolumeclaims "test.csi.azure.comlznpd" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
STEP: Deleting pvc
Oct  9 03:08:04.701: INFO: Deleting PersistentVolumeClaim "test.csi.azure.comlznpd"
Oct  9 03:08:04.813: INFO: Waiting up to 5m0s for PersistentVolume pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078 to get deleted
Oct  9 03:08:04.923: INFO: PersistentVolume pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078 found and phase=Released (109.945003ms)
Oct  9 03:08:10.041: INFO: PersistentVolume pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078 found and phase=Released (5.227909223s)
Oct  9 03:08:15.154: INFO: PersistentVolume pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078 was removed
... skipping 9 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not allow expansion of pvcs without AllowVolumeExpansion property
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:157
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property","total":9,"completed":2,"skipped":522,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
... skipping 128 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":19,"completed":6,"skipped":688,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral 
  should create read-only inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
... skipping 43 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume","total":19,"completed":7,"skipped":699,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Oct  9 03:09:57.924: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 116 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumes should store data","total":19,"completed":8,"skipped":810,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
... skipping 67 lines ...
Oct  9 03:11:32.150: INFO: Running AfterSuite actions on node 1



Summarizing 3 Failures:

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [Measurement] should support two pods which share the same volume 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:201

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral [Measurement] should support two pods which share the same volume 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:321

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [Measurement] should create read-only inline ephemeral volume 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:321

Ran 34 of 6664 Specs in 1801.452 seconds
FAIL! -- 31 Passed | 3 Failed | 0 Pending | 6630 Skipped 

Ginkgo ran 1 suite in 30m5.713738106s
Test Suite Failed
+ print_logs
+ sed -i s/disk.csi.azure.com/test.csi.azure.com/g deploy/example/storageclass-azuredisk-csi.yaml
+ bash ./hack/verify-examples.sh linux azurepubliccloud ephemeral test
begin to create deployment examples ...
storageclass.storage.k8s.io/managed-csi created
Applying config "deploy/example/deployment.yaml"
... skipping 73 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I1009 02:41:23.108924       1 azuredisk.go:139] driver userAgent: test.csi.azure.com/v1.8.0-9eaf1e00dfb23da1f5430a51a79908d1683f455a gc/go1.17.1 (amd64-linux) e2e-test
I1009 02:41:23.110146       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1009 02:41:23.131403       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1009 02:41:23.131445       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1009 02:41:23.131470       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1009 02:41:23.131501       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1009 02:41:23.200075       1 azure_auth.go:234] Using AzurePublicCloud environment
I1009 02:41:23.200107       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I1009 02:41:23.200113       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 87 lines ...
I1009 02:41:24.270676       1 utils.go:96] GRPC request: {}
I1009 02:41:24.270699       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:41:33.426486       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:33.426534       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-165d8edd-c536-4637-826e-600f10d4255a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-165d8edd-c536-4637-826e-600f10d4255a","csi.storage.k8s.io/pvc/name":"test.csi.azure.comxjffk","csi.storage.k8s.io/pvc/namespace":"volumemode-4807"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:33.559751       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:33.559784       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcxtfr","csi.storage.k8s.io/pvc/namespace":"volume-expand-3165"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1009 02:41:33.741674       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:33.741835       1 controllerserver.go:271] begin to create azure disk(pvc-165d8edd-c536-4637-826e-600f10d4255a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:33.741856       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-165d8edd-c536-4637-826e-600f10d4255a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:33.799780       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:33.799896       1 controllerserver.go:271] begin to create azure disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:33.799914       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:34.013706       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:34.013733       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-2eff62da-3354-4f5b-9301-00fde723174a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2eff62da-3354-4f5b-9301-00fde723174a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-m9mf7-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-4559"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:34.100276       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-2eff62da-3354-4f5b-9301-00fde723174a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-2eff62da-3354-4f5b-9301-00fde723174a' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:34.100366       1 controllerserver.go:271] begin to create azure disk(pvc-2eff62da-3354-4f5b-9301-00fde723174a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:34.100382       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-2eff62da-3354-4f5b-9301-00fde723174a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:34.486403       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:34.486429       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a1efada0-40c8-497a-9795-6ed4682d940e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a1efada0-40c8-497a-9795-6ed4682d940e","csi.storage.k8s.io/pvc/name":"test.csi.azure.com2tcvt","csi.storage.k8s.io/pvc/namespace":"volume-6373"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:34.505481       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:34.505566       1 controllerserver.go:271] begin to create azure disk(pvc-a1efada0-40c8-497a-9795-6ed4682d940e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:34.505579       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a1efada0-40c8-497a-9795-6ed4682d940e StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:34.694870       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:34.694891       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e","csi.storage.k8s.io/pvc/name":"test.csi.azure.compgjr8","csi.storage.k8s.io/pvc/namespace":"volumemode-8971"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1009 02:41:34.747479       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:34.747572       1 controllerserver.go:271] begin to create azure disk(pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:34.747590       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:35.254275       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:35.254301       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-0c30bc9f-852c-48a5-8564-a4a33da18284","parameters":{"csi.storage.k8s.io/pv/name":"pvc-0c30bc9f-852c-48a5-8564-a4a33da18284","csi.storage.k8s.io/pvc/name":"test.csi.azure.comhk9b6","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:35.281366       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0c30bc9f-852c-48a5-8564-a4a33da18284, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-0c30bc9f-852c-48a5-8564-a4a33da18284' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:35.281442       1 controllerserver.go:271] begin to create azure disk(pvc-0c30bc9f-852c-48a5-8564-a4a33da18284) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:35.281464       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-0c30bc9f-852c-48a5-8564-a4a33da18284 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:36.084399       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:36.084441       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f","csi.storage.k8s.io/pvc/name":"inline-volume-tester-w2csj-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:36.103513       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:36.103599       1 controllerserver.go:271] begin to create azure disk(pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:36.103631       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:36.173687       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-165d8edd-c536-4637-826e-600f10d4255a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:36.173742       1 controllerserver.go:348] create azure disk(pvc-165d8edd-c536-4637-826e-600f10d4255a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-165d8edd-c536-4637-826e-600f10d4255a kubernetes.io-created-for-pvc-name:test.csi.azure.comxjffk kubernetes.io-created-for-pvc-namespace:volumemode-4807]) successfully
I1009 02:41:36.173785       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-165d8edd-c536-4637-826e-600f10d4255a","csi.storage.k8s.io/pvc/name":"test.csi.azure.comxjffk","csi.storage.k8s.io/pvc/namespace":"volumemode-4807","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a"}}
I1009 02:41:36.177571       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb StorageAccountType:StandardSSD_LRS Size:5
... skipping 13 lines ...
I1009 02:41:37.611906       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-0c30bc9f-852c-48a5-8564-a4a33da18284","csi.storage.k8s.io/pvc/name":"test.csi.azure.comhk9b6","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0c30bc9f-852c-48a5-8564-a4a33da18284"}}
I1009 02:41:38.480068       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:38.480114       1 controllerserver.go:348] create azure disk(pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f kubernetes.io-created-for-pvc-name:inline-volume-tester-w2csj-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-5454]) successfully
I1009 02:41:38.480138       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f","csi.storage.k8s.io/pvc/name":"inline-volume-tester-w2csj-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f"}}
I1009 02:41:39.912514       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:39.912543       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153","parameters":{"csi.storage.k8s.io/pv/name":"pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153","csi.storage.k8s.io/pvc/name":"pvc-zjcmw","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:39.931324       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:39.931419       1 controllerserver.go:271] begin to create azure disk(pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:39.931437       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.022129       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.022156       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-e8570a05-f168-4387-bab6-c00121b30c84","parameters":{"csi.storage.k8s.io/pv/name":"pvc-e8570a05-f168-4387-bab6-c00121b30c84","csi.storage.k8s.io/pvc/name":"pvc-sllrp","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.050849       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.050925       1 controllerserver.go:271] begin to create azure disk(pvc-e8570a05-f168-4387-bab6-c00121b30c84) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.050943       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-e8570a05-f168-4387-bab6-c00121b30c84 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.142108       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.142133       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431","parameters":{"csi.storage.k8s.io/pv/name":"pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431","csi.storage.k8s.io/pvc/name":"pvc-8f7jm","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.176887       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.176964       1 controllerserver.go:271] begin to create azure disk(pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.176981       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.254024       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.254051       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2","csi.storage.k8s.io/pvc/name":"pvc-9h4sb","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.321356       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.321470       1 controllerserver.go:271] begin to create azure disk(pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.321505       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.365271       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.365300       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7","csi.storage.k8s.io/pvc/name":"pvc-kpdqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.462574       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.462659       1 controllerserver.go:271] begin to create azure disk(pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.462676       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.500599       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.500627       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b6307f77-27bf-4d29-a99b-6656c918312f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b6307f77-27bf-4d29-a99b-6656c918312f","csi.storage.k8s.io/pvc/name":"pvc-wfz7b","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.518256       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.518334       1 controllerserver.go:271] begin to create azure disk(pvc-b6307f77-27bf-4d29-a99b-6656c918312f) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.518359       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b6307f77-27bf-4d29-a99b-6656c918312f StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.586592       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.586621       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-3f30ec72-2180-4324-af67-1c21807af88e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-3f30ec72-2180-4324-af67-1c21807af88e","csi.storage.k8s.io/pvc/name":"pvc-wpptx","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.605872       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.606003       1 controllerserver.go:271] begin to create azure disk(pvc-3f30ec72-2180-4324-af67-1c21807af88e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.606020       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-3f30ec72-2180-4324-af67-1c21807af88e StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.695661       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.695687       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a","csi.storage.k8s.io/pvc/name":"pvc-r8z8x","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.715479       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.715564       1 controllerserver.go:271] begin to create azure disk(pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.715592       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.807072       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.807099       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a","csi.storage.k8s.io/pvc/name":"pvc-66t68","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.827422       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.827503       1 controllerserver.go:271] begin to create azure disk(pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.827528       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:40.918370       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:40.918396       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38","csi.storage.k8s.io/pvc/name":"pvc-8qt5x","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:40.955946       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:40.956025       1 controllerserver.go:271] begin to create azure disk(pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:40.956042       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.030132       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.030159       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-bdc2dd19-74a6-43e0-b235-d251154ece04","parameters":{"csi.storage.k8s.io/pv/name":"pvc-bdc2dd19-74a6-43e0-b235-d251154ece04","csi.storage.k8s.io/pvc/name":"pvc-tpnhb","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.074121       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.074202       1 controllerserver.go:271] begin to create azure disk(pvc-bdc2dd19-74a6-43e0-b235-d251154ece04) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.074220       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-bdc2dd19-74a6-43e0-b235-d251154ece04 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.141417       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.141444       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb","csi.storage.k8s.io/pvc/name":"pvc-5zqqk","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.161802       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.161879       1 controllerserver.go:271] begin to create azure disk(pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.161895       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.250421       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.250448       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-77114e1d-ced2-44d6-b825-6e13cb80e212","parameters":{"csi.storage.k8s.io/pv/name":"pvc-77114e1d-ced2-44d6-b825-6e13cb80e212","csi.storage.k8s.io/pvc/name":"pvc-bnn7z","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.269439       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.269519       1 controllerserver.go:271] begin to create azure disk(pvc-77114e1d-ced2-44d6-b825-6e13cb80e212) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.269537       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-77114e1d-ced2-44d6-b825-6e13cb80e212 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.361678       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.361706       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7","csi.storage.k8s.io/pvc/name":"pvc-85jxm","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.385650       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.385735       1 controllerserver.go:271] begin to create azure disk(pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.385752       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.471237       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.471267       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b6cf353d-f05a-49be-8090-56795f944cf4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b6cf353d-f05a-49be-8090-56795f944cf4","csi.storage.k8s.io/pvc/name":"pvc-h54xk","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.490548       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.490629       1 controllerserver.go:271] begin to create azure disk(pvc-b6cf353d-f05a-49be-8090-56795f944cf4) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.490646       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b6cf353d-f05a-49be-8090-56795f944cf4 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.580632       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.580660       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-6df84773-5354-4790-958d-beb160084ccd","parameters":{"csi.storage.k8s.io/pv/name":"pvc-6df84773-5354-4790-958d-beb160084ccd","csi.storage.k8s.io/pvc/name":"pvc-bbs94","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.600456       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.600535       1 controllerserver.go:271] begin to create azure disk(pvc-6df84773-5354-4790-958d-beb160084ccd) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.600585       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-6df84773-5354-4790-958d-beb160084ccd StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.690728       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.690755       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-f39fe354-8111-4de5-b660-c804ed08f0de","parameters":{"csi.storage.k8s.io/pv/name":"pvc-f39fe354-8111-4de5-b660-c804ed08f0de","csi.storage.k8s.io/pvc/name":"pvc-xhn45","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.710408       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.710473       1 controllerserver.go:271] begin to create azure disk(pvc-f39fe354-8111-4de5-b660-c804ed08f0de) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.710489       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-f39fe354-8111-4de5-b660-c804ed08f0de StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.800008       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.800035       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d","csi.storage.k8s.io/pvc/name":"pvc-2djxj","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.849840       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.849961       1 controllerserver.go:271] begin to create azure disk(pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.849981       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:41.910847       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:41.910875       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c","csi.storage.k8s.io/pvc/name":"pvc-2fs5n","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:41.944723       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:41.944808       1 controllerserver.go:271] begin to create azure disk(pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:41.944827       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.020370       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.020396       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-39b3fa75-3672-44d3-80aa-5c8b181102df","parameters":{"csi.storage.k8s.io/pv/name":"pvc-39b3fa75-3672-44d3-80aa-5c8b181102df","csi.storage.k8s.io/pvc/name":"pvc-hk7wr","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.048735       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.048808       1 controllerserver.go:271] begin to create azure disk(pvc-39b3fa75-3672-44d3-80aa-5c8b181102df) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.048825       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-39b3fa75-3672-44d3-80aa-5c8b181102df StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.134920       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.134946       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4","csi.storage.k8s.io/pvc/name":"pvc-t7nhc","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.155502       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.155578       1 controllerserver.go:271] begin to create azure disk(pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.155594       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.249302       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.249329       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-21c3a0e3-938f-4782-b96c-5133045752b8","parameters":{"csi.storage.k8s.io/pv/name":"pvc-21c3a0e3-938f-4782-b96c-5133045752b8","csi.storage.k8s.io/pvc/name":"pvc-jlxpl","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.268792       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.268880       1 controllerserver.go:271] begin to create azure disk(pvc-21c3a0e3-938f-4782-b96c-5133045752b8) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.268908       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-21c3a0e3-938f-4782-b96c-5133045752b8 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.341680       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.341721       1 controllerserver.go:348] create azure disk(pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153 kubernetes.io-created-for-pvc-name:pvc-zjcmw kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:42.341749       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153","csi.storage.k8s.io/pvc/name":"pvc-zjcmw","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153"}}
I1009 02:41:42.356336       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.356403       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e","csi.storage.k8s.io/pvc/name":"pvc-bqbw4","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.376221       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.376287       1 controllerserver.go:271] begin to create azure disk(pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.376309       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.418005       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-e8570a05-f168-4387-bab6-c00121b30c84 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.418047       1 controllerserver.go:348] create azure disk(pvc-e8570a05-f168-4387-bab6-c00121b30c84) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-e8570a05-f168-4387-bab6-c00121b30c84 kubernetes.io-created-for-pvc-name:pvc-sllrp kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:42.418071       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e8570a05-f168-4387-bab6-c00121b30c84","csi.storage.k8s.io/pvc/name":"pvc-sllrp","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84"}}
I1009 02:41:42.467194       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.467219       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df","parameters":{"csi.storage.k8s.io/pv/name":"pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df","csi.storage.k8s.io/pvc/name":"pvc-s87rd","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.485051       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.485134       1 controllerserver.go:271] begin to create azure disk(pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.485153       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.500073       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.500115       1 controllerserver.go:348] create azure disk(pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431 kubernetes.io-created-for-pvc-name:pvc-8f7jm kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:42.500141       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431","csi.storage.k8s.io/pvc/name":"pvc-8f7jm","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431"}}
I1009 02:41:42.577877       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.577910       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24","parameters":{"csi.storage.k8s.io/pv/name":"pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24","csi.storage.k8s.io/pvc/name":"pvc-4x299","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.597288       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.597359       1 controllerserver.go:271] begin to create azure disk(pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.597374       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.685790       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.685828       1 controllerserver.go:348] create azure disk(pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2 kubernetes.io-created-for-pvc-name:pvc-9h4sb kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:42.685853       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2","csi.storage.k8s.io/pvc/name":"pvc-9h4sb","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2"}}
I1009 02:41:42.687135       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.687155       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09","csi.storage.k8s.io/pvc/name":"pvc-lmlfs","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.708691       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.708759       1 controllerserver.go:271] begin to create azure disk(pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.708774       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.797033       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.797060       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-5878152e-4c5d-4ae9-9862-d45656993cae","parameters":{"csi.storage.k8s.io/pv/name":"pvc-5878152e-4c5d-4ae9-9862-d45656993cae","csi.storage.k8s.io/pvc/name":"pvc-lhtw6","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.799086       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.799122       1 controllerserver.go:348] create azure disk(pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7 kubernetes.io-created-for-pvc-name:pvc-kpdqs kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:42.799144       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7","csi.storage.k8s.io/pvc/name":"pvc-kpdqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7"}}
I1009 02:41:42.816524       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.816592       1 controllerserver.go:271] begin to create azure disk(pvc-5878152e-4c5d-4ae9-9862-d45656993cae) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.816626       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-5878152e-4c5d-4ae9-9862-d45656993cae StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.836275       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-b6307f77-27bf-4d29-a99b-6656c918312f StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.836316       1 controllerserver.go:348] create azure disk(pvc-b6307f77-27bf-4d29-a99b-6656c918312f) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-b6307f77-27bf-4d29-a99b-6656c918312f kubernetes.io-created-for-pvc-name:pvc-wfz7b kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:42.836340       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b6307f77-27bf-4d29-a99b-6656c918312f","csi.storage.k8s.io/pvc/name":"pvc-wfz7b","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f"}}
I1009 02:41:42.907108       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:42.907139       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-15c1b3cd-19a6-4076-b236-6fb749963aca","parameters":{"csi.storage.k8s.io/pv/name":"pvc-15c1b3cd-19a6-4076-b236-6fb749963aca","csi.storage.k8s.io/pvc/name":"pvc-t2kjm","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:42.926848       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:42.926918       1 controllerserver.go:271] begin to create azure disk(pvc-15c1b3cd-19a6-4076-b236-6fb749963aca) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:42.926934       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-15c1b3cd-19a6-4076-b236-6fb749963aca StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.973462       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-3f30ec72-2180-4324-af67-1c21807af88e StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:42.973511       1 controllerserver.go:348] create azure disk(pvc-3f30ec72-2180-4324-af67-1c21807af88e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-3f30ec72-2180-4324-af67-1c21807af88e kubernetes.io-created-for-pvc-name:pvc-wpptx kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:42.973537       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3f30ec72-2180-4324-af67-1c21807af88e","csi.storage.k8s.io/pvc/name":"pvc-wpptx","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e"}}
I1009 02:41:43.017873       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:43.017899       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198","csi.storage.k8s.io/pvc/name":"pvc-7xjp6","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:43.036148       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:43.036212       1 controllerserver.go:271] begin to create azure disk(pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:43.036228       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.037893       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.037926       1 controllerserver.go:348] create azure disk(pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a kubernetes.io-created-for-pvc-name:pvc-r8z8x kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:43.037947       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a","csi.storage.k8s.io/pvc/name":"pvc-r8z8x","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a"}}
I1009 02:41:43.134222       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:43.134249       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9de70f81-5326-4e06-9cba-36c71821fbe2","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9de70f81-5326-4e06-9cba-36c71821fbe2","csi.storage.k8s.io/pvc/name":"pvc-v8nzz","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:43.154552       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:43.154631       1 controllerserver.go:271] begin to create azure disk(pvc-9de70f81-5326-4e06-9cba-36c71821fbe2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:43.154647       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9de70f81-5326-4e06-9cba-36c71821fbe2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.236701       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:43.236727       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2","parameters":{"csi.storage.k8s.io/pv/name":"pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2","csi.storage.k8s.io/pvc/name":"pvc-cdtbd","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:43.256513       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.256556       1 controllerserver.go:348] create azure disk(pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38 kubernetes.io-created-for-pvc-name:pvc-8qt5x kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:43.256579       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38","csi.storage.k8s.io/pvc/name":"pvc-8qt5x","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38"}}
I1009 02:41:43.257198       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:43.257260       1 controllerserver.go:271] begin to create azure disk(pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:43.257289       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.266292       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.266327       1 controllerserver.go:348] create azure disk(pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a kubernetes.io-created-for-pvc-name:pvc-66t68 kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:43.266347       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a","csi.storage.k8s.io/pvc/name":"pvc-66t68","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a"}}
I1009 02:41:43.357292       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:41:43.357320       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734","csi.storage.k8s.io/pvc/name":"pvc-4pp69","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:41:43.370715       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-bdc2dd19-74a6-43e0-b235-d251154ece04 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.370757       1 controllerserver.go:348] create azure disk(pvc-bdc2dd19-74a6-43e0-b235-d251154ece04) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-bdc2dd19-74a6-43e0-b235-d251154ece04 kubernetes.io-created-for-pvc-name:pvc-tpnhb kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:43.370782       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-bdc2dd19-74a6-43e0-b235-d251154ece04","csi.storage.k8s.io/pvc/name":"pvc-tpnhb","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04"}}
I1009 02:41:43.377350       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:41:43.377419       1 controllerserver.go:271] begin to create azure disk(pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:41:43.377436       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.495328       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:43.495365       1 controllerserver.go:348] create azure disk(pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb kubernetes.io-created-for-pvc-name:pvc-5zqqk kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:43.495388       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb","csi.storage.k8s.io/pvc/name":"pvc-5zqqk","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb"}}
I1009 02:41:43.596456       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-77114e1d-ced2-44d6-b825-6e13cb80e212 StorageAccountType:StandardSSD_LRS Size:5
... skipping 55 lines ...
I1009 02:41:45.541538       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2","csi.storage.k8s.io/pvc/name":"pvc-cdtbd","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2"}}
I1009 02:41:45.709094       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:41:45.709137       1 controllerserver.go:348] create azure disk(pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734 kubernetes.io-created-for-pvc-name:pvc-4pp69 kubernetes.io-created-for-pvc-namespace:volumelimits-9466]) successfully
I1009 02:41:45.709161       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734","csi.storage.k8s.io/pvc/name":"pvc-4pp69","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734"}}
I1009 02:42:00.074736       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:42:00.074765       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-aaec181d-b96b-4280-9361-eab52a2a07d5","parameters":{"csi.storage.k8s.io/pv/name":"pvc-aaec181d-b96b-4280-9361-eab52a2a07d5","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-jmvp4-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:42:00.094346       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:42:00.094425       1 controllerserver.go:271] begin to create azure disk(pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:42:00.094444       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-aaec181d-b96b-4280-9361-eab52a2a07d5 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:42:02.397401       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-aaec181d-b96b-4280-9361-eab52a2a07d5 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:42:02.397442       1 controllerserver.go:348] create azure disk(pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-aaec181d-b96b-4280-9361-eab52a2a07d5 kubernetes.io-created-for-pvc-name:inline-volume-tester2-jmvp4-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-5454]) successfully
I1009 02:42:02.397466       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-aaec181d-b96b-4280-9361-eab52a2a07d5","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-jmvp4-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5"}}
I1009 02:42:03.108014       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 02:42:03.108038       1 utils.go:96] GRPC request: {}
I1009 02:42:03.108105       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:42:03.108872       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 02:42:03.108896       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:42:03.139346       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6442450944)
I1009 02:42:03.161699       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 02:42:03.161784       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 02:42:04.169003       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 02:42:04.169039       1 utils.go:96] GRPC request: {}
I1009 02:42:04.169092       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:42:04.169641       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 02:42:04.169657       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:42:04.192470       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6442450944)
I1009 02:42:04.213984       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 02:42:04.214059       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 02:42:06.221537       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 02:42:06.221585       1 utils.go:96] GRPC request: {}
I1009 02:42:06.221629       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:42:06.222160       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 02:42:06.222177       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:42:06.245275       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6442450944)
I1009 02:42:06.268125       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 02:42:06.268210       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 02:42:06.275345       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 02:42:06.275366       1 utils.go:96] GRPC request: {}
I1009 02:42:06.275405       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:42:06.275974       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 02:42:06.275993       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:42:06.304370       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6442450944)
I1009 02:42:06.327192       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 02:42:06.327268       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 02:42:10.275435       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 02:42:10.275466       1 utils.go:96] GRPC request: {}
I1009 02:42:10.275513       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:42:10.276116       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 02:42:10.276137       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:42:10.301230       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6442450944)
I1009 02:42:10.325267       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 02:42:10.325360       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 02:42:10.329812       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 02:42:10.329833       1 utils.go:96] GRPC request: {}
I1009 02:42:10.329876       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:42:10.330425       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 02:42:10.330443       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:42:10.361645       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6442450944)
I1009 02:42:10.384055       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 02:42:10.384100       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 02:42:26.339413       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 02:42:26.339441       1 utils.go:96] GRPC request: {}
I1009 02:42:26.339486       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 02:42:26.340102       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 02:42:26.340122       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:42:26.385785       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6442450944)
I1009 02:42:26.412090       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 02:42:26.412138       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 02:42:57.022380       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:42:57.022411       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a"}
I1009 02:42:57.022494       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a)
I1009 02:43:02.243413       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a
I1009 02:43:02.243442       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a) returned with <nil>
I1009 02:43:02.243464       1 utils.go:102] GRPC response: {}
I1009 02:43:06.227770       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:43:06.227801       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e"}
I1009 02:43:06.227903       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e)
I1009 02:43:09.340821       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:43:09.340846       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6","parameters":{"csi.storage.k8s.io/pv/name":"pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6","csi.storage.k8s.io/pvc/name":"test.csi.azure.comf5kvn","csi.storage.k8s.io/pvc/namespace":"provisioning-99"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:43:09.361367       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:43:09.361440       1 controllerserver.go:271] begin to create azure disk(pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:43:09.361460       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:43:11.440300       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e
I1009 02:43:11.440328       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e) returned with <nil>
I1009 02:43:11.440344       1 utils.go:102] GRPC response: {}
I1009 02:43:11.643440       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:43:11.643498       1 controllerserver.go:348] create azure disk(pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6 kubernetes.io-created-for-pvc-name:test.csi.azure.comf5kvn kubernetes.io-created-for-pvc-namespace:provisioning-99]) successfully
I1009 02:43:11.643520       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6","csi.storage.k8s.io/pvc/name":"test.csi.azure.comf5kvn","csi.storage.k8s.io/pvc/namespace":"provisioning-99","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6"}}
I1009 02:43:16.270780       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:43:16.270809       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47","parameters":{"csi.storage.k8s.io/pv/name":"pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47","csi.storage.k8s.io/pvc/name":"test.csi.azure.comzx4ds","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5253"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:43:16.290045       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:43:16.290113       1 controllerserver.go:271] begin to create azure disk(pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:43:16.290130       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:43:18.808736       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:43:18.808781       1 controllerserver.go:348] create azure disk(pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47 kubernetes.io-created-for-pvc-name:test.csi.azure.comzx4ds kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-5253]) successfully
I1009 02:43:18.808806       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47","csi.storage.k8s.io/pvc/name":"test.csi.azure.comzx4ds","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5253","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47"}}
I1009 02:43:30.420662       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
... skipping 14 lines ...
I1009 02:43:54.917643       1 utils.go:102] GRPC response: {}
I1009 02:44:02.265861       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:44:02.265889       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-62787193-2db7-4abf-b655-aef7e0cbc306","parameters":{"csi.storage.k8s.io/pv/name":"pvc-62787193-2db7-4abf-b655-aef7e0cbc306","csi.storage.k8s.io/pvc/name":"test.csi.azure.comw6gwl","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5157"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:44:02.280216       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:44:02.280237       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e"}
I1009 02:44:02.280308       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e)
I1009 02:44:02.287581       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:44:02.287647       1 controllerserver.go:271] begin to create azure disk(pvc-62787193-2db7-4abf-b655-aef7e0cbc306) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:44:02.287664       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-62787193-2db7-4abf-b655-aef7e0cbc306 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:44:04.661323       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-62787193-2db7-4abf-b655-aef7e0cbc306 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:44:04.661382       1 controllerserver.go:348] create azure disk(pvc-62787193-2db7-4abf-b655-aef7e0cbc306) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-62787193-2db7-4abf-b655-aef7e0cbc306 kubernetes.io-created-for-pvc-name:test.csi.azure.comw6gwl kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-5157]) successfully
I1009 02:44:04.661441       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-62787193-2db7-4abf-b655-aef7e0cbc306","csi.storage.k8s.io/pvc/name":"test.csi.azure.comw6gwl","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5157","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306"}}
I1009 02:44:07.456553       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e
I1009 02:44:07.456587       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e) returned with <nil>
I1009 02:44:07.456604       1 utils.go:102] GRPC response: {}
I1009 02:44:10.053914       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:44:10.053941       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b19575ad-ba29-4526-b1d1-8999de9c383e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b19575ad-ba29-4526-b1d1-8999de9c383e","csi.storage.k8s.io/pvc/name":"test.csi.azure.com662gm","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-2142"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:44:10.092026       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:44:10.092109       1 controllerserver.go:271] begin to create azure disk(pvc-b19575ad-ba29-4526-b1d1-8999de9c383e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:44:10.092127       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b19575ad-ba29-4526-b1d1-8999de9c383e StorageAccountType:StandardSSD_LRS Size:5
I1009 02:44:12.425508       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-b19575ad-ba29-4526-b1d1-8999de9c383e StorageAccountType:StandardSSD_LRS Size:5
I1009 02:44:12.425565       1 controllerserver.go:348] create azure disk(pvc-b19575ad-ba29-4526-b1d1-8999de9c383e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-b19575ad-ba29-4526-b1d1-8999de9c383e kubernetes.io-created-for-pvc-name:test.csi.azure.com662gm kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-2142]) successfully
I1009 02:44:12.425589       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b19575ad-ba29-4526-b1d1-8999de9c383e","csi.storage.k8s.io/pvc/name":"test.csi.azure.com662gm","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-2142","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e"}}
I1009 02:44:46.781518       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:44:46.781548       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47"}
I1009 02:44:46.781626       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47)
I1009 02:44:51.961541       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47
I1009 02:44:51.961574       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47) returned with <nil>
I1009 02:44:51.961600       1 utils.go:102] GRPC response: {}
I1009 02:44:58.532815       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:44:58.532843       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-a9d24570-ad6a-4920-a8de-d80c64329714","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a9d24570-ad6a-4920-a8de-d80c64329714","csi.storage.k8s.io/pvc/name":"pvc-sbvfm","csi.storage.k8s.io/pvc/namespace":"provisioning-6579"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1009 02:44:58.552406       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:44:58.552530       1 controllerserver.go:271] begin to create azure disk(pvc-a9d24570-ad6a-4920-a8de-d80c64329714) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:44:58.552548       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a9d24570-ad6a-4920-a8de-d80c64329714 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:00.937570       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a9d24570-ad6a-4920-a8de-d80c64329714 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:00.937625       1 controllerserver.go:348] create azure disk(pvc-a9d24570-ad6a-4920-a8de-d80c64329714) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-a9d24570-ad6a-4920-a8de-d80c64329714 kubernetes.io-created-for-pvc-name:pvc-sbvfm kubernetes.io-created-for-pvc-namespace:provisioning-6579]) successfully
I1009 02:45:00.937668       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a9d24570-ad6a-4920-a8de-d80c64329714","csi.storage.k8s.io/pvc/name":"pvc-sbvfm","csi.storage.k8s.io/pvc/namespace":"provisioning-6579","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714"}}
I1009 02:45:17.824794       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 13 lines ...
I1009 02:45:26.134625       1 utils.go:102] GRPC response: {}
I1009 02:45:26.247670       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306
I1009 02:45:26.247701       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306) returned with <nil>
I1009 02:45:26.247718       1 utils.go:102] GRPC response: {}
I1009 02:45:28.473228       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:45:28.473252       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526","csi.storage.k8s.io/pvc/name":"test.csi.azure.com22mhx","csi.storage.k8s.io/pvc/namespace":"provisioning-6500"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:45:28.502758       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:45:28.502839       1 controllerserver.go:271] begin to create azure disk(pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:45:28.502861       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:30.836109       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:30.836181       1 controllerserver.go:348] create azure disk(pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526 kubernetes.io-created-for-pvc-name:test.csi.azure.com22mhx kubernetes.io-created-for-pvc-namespace:provisioning-6500]) successfully
I1009 02:45:30.836235       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526","csi.storage.k8s.io/pvc/name":"test.csi.azure.com22mhx","csi.storage.k8s.io/pvc/namespace":"provisioning-6500","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526"}}
I1009 02:45:33.026629       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:45:33.026660       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-hn872-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-7339"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:45:33.045104       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:45:33.045196       1 controllerserver.go:271] begin to create azure disk(pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:45:33.045214       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:34.214653       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:45:34.214680       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qwk8d-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-526"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:45:34.234680       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:45:34.234765       1 controllerserver.go:271] begin to create azure disk(pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:45:34.234780       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:35.428279       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:35.428320       1 controllerserver.go:348] create azure disk(pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a kubernetes.io-created-for-pvc-name:inline-volume-tester-hn872-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-7339]) successfully
I1009 02:45:35.428343       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-hn872-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-7339","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a"}}
I1009 02:45:36.537082       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d StorageAccountType:StandardSSD_LRS Size:5
... skipping 6 lines ...
I1009 02:45:50.006575       1 utils.go:96] GRPC request: {"name":"snapshot-60c8d452-5451-4d88-88de-2f7a66ab7de5","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714"}
I1009 02:45:50.006658       1 controllerserver.go:909] begin to create snapshot(snapshot-60c8d452-5451-4d88-88de-2f7a66ab7de5, incremental: true) under rg(kubetest-enl5pco2)
I1009 02:45:52.442925       1 controllerserver.go:918] create snapshot(snapshot-60c8d452-5451-4d88-88de-2f7a66ab7de5) under rg(kubetest-enl5pco2) successfully
I1009 02:45:52.465023       1 utils.go:102] GRPC response: {"snapshot":{"creation_time":{"nanos":182119100,"seconds":1633747550},"ready_to_use":true,"size_bytes":5368709120,"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/snapshots/snapshot-60c8d452-5451-4d88-88de-2f7a66ab7de5","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714"}}
I1009 02:45:54.688190       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:45:54.688222       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-253a80cc-1fe0-4e8b-a303-f90c10475177","parameters":{"csi.storage.k8s.io/pv/name":"pvc-253a80cc-1fe0-4e8b-a303-f90c10475177","csi.storage.k8s.io/pvc/name":"pvc-s6wlp","csi.storage.k8s.io/pvc/namespace":"provisioning-6579"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}],"volume_content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/snapshots/snapshot-60c8d452-5451-4d88-88de-2f7a66ab7de5"}}}}
I1009 02:45:54.707762       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-253a80cc-1fe0-4e8b-a303-f90c10475177, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-253a80cc-1fe0-4e8b-a303-f90c10475177' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:45:54.707854       1 controllerserver.go:271] begin to create azure disk(pvc-253a80cc-1fe0-4e8b-a303-f90c10475177) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:45:54.707892       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-253a80cc-1fe0-4e8b-a303-f90c10475177 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:57.059076       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-253a80cc-1fe0-4e8b-a303-f90c10475177 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:45:57.059125       1 controllerserver.go:348] create azure disk(pvc-253a80cc-1fe0-4e8b-a303-f90c10475177) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-253a80cc-1fe0-4e8b-a303-f90c10475177 kubernetes.io-created-for-pvc-name:pvc-s6wlp kubernetes.io-created-for-pvc-namespace:provisioning-6579]) successfully
I1009 02:45:57.059151       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/snapshots/snapshot-60c8d452-5451-4d88-88de-2f7a66ab7de5"}}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-253a80cc-1fe0-4e8b-a303-f90c10475177","csi.storage.k8s.io/pvc/name":"pvc-s6wlp","csi.storage.k8s.io/pvc/namespace":"provisioning-6579","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-253a80cc-1fe0-4e8b-a303-f90c10475177"}}
I1009 02:47:14.079195       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 12 lines ...
I1009 02:47:29.501229       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714) returned with <nil>
I1009 02:47:29.501246       1 utils.go:102] GRPC response: {}
I1009 02:47:29.841625       1 controllerserver.go:961] delete snapshot(snapshot-60c8d452-5451-4d88-88de-2f7a66ab7de5) under rg(kubetest-enl5pco2) successfully
I1009 02:47:29.841666       1 utils.go:102] GRPC response: {}
I1009 02:47:32.973672       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:47:32.973703       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-f8058fe6-61ba-4b61-a829-20311e63f16d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-f8058fe6-61ba-4b61-a829-20311e63f16d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comb8lxw","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-4743"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:47:32.992952       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:47:32.993062       1 controllerserver.go:271] begin to create azure disk(pvc-f8058fe6-61ba-4b61-a829-20311e63f16d) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:47:32.993078       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-f8058fe6-61ba-4b61-a829-20311e63f16d StorageAccountType:StandardSSD_LRS Size:5
I1009 02:47:35.312581       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-f8058fe6-61ba-4b61-a829-20311e63f16d StorageAccountType:StandardSSD_LRS Size:5
I1009 02:47:35.312625       1 controllerserver.go:348] create azure disk(pvc-f8058fe6-61ba-4b61-a829-20311e63f16d) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-f8058fe6-61ba-4b61-a829-20311e63f16d kubernetes.io-created-for-pvc-name:test.csi.azure.comb8lxw kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-4743]) successfully
I1009 02:47:35.312648       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f8058fe6-61ba-4b61-a829-20311e63f16d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comb8lxw","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-4743","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d"}}
I1009 02:47:48.801578       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 4 lines ...
I1009 02:47:54.034419       1 utils.go:102] GRPC response: {}
I1009 02:47:54.548362       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:47:54.548397       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e"}
I1009 02:47:54.548485       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e)
I1009 02:47:58.832390       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:47:58.832417       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2","parameters":{"csi.storage.k8s.io/pv/name":"pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2","csi.storage.k8s.io/pvc/name":"test.csi.azure.comhlg2g","csi.storage.k8s.io/pvc/namespace":"volume-1889"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:47:58.854157       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:47:58.854281       1 controllerserver.go:271] begin to create azure disk(pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:47:58.854298       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:47:59.765704       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e
I1009 02:47:59.765737       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e) returned with <nil>
I1009 02:47:59.765756       1 utils.go:102] GRPC response: {}
I1009 02:48:01.161756       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:48:01.161799       1 controllerserver.go:348] create azure disk(pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2 kubernetes.io-created-for-pvc-name:test.csi.azure.comhlg2g kubernetes.io-created-for-pvc-namespace:volume-1889]) successfully
I1009 02:48:01.161823       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2","csi.storage.k8s.io/pvc/name":"test.csi.azure.comhlg2g","csi.storage.k8s.io/pvc/namespace":"volume-1889","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2"}}
I1009 02:48:05.872727       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:48:05.872755       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad","parameters":{"csi.storage.k8s.io/pv/name":"pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad","csi.storage.k8s.io/pvc/name":"test.csi.azure.com8d28f","csi.storage.k8s.io/pvc/namespace":"provisioning-5586"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:48:05.892630       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:48:05.892746       1 controllerserver.go:271] begin to create azure disk(pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:48:05.892761       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad StorageAccountType:StandardSSD_LRS Size:5
I1009 02:48:08.209311       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad StorageAccountType:StandardSSD_LRS Size:5
I1009 02:48:08.209354       1 controllerserver.go:348] create azure disk(pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad kubernetes.io-created-for-pvc-name:test.csi.azure.com8d28f kubernetes.io-created-for-pvc-namespace:provisioning-5586]) successfully
I1009 02:48:08.209377       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad","csi.storage.k8s.io/pvc/name":"test.csi.azure.com8d28f","csi.storage.k8s.io/pvc/namespace":"provisioning-5586","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad"}}
I1009 02:49:03.550503       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 7 lines ...
I1009 02:49:43.522348       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d)
I1009 02:49:48.766010       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d
I1009 02:49:48.766044       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d) returned with <nil>
I1009 02:49:48.766061       1 utils.go:102] GRPC response: {}
I1009 02:49:53.654145       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:49:53.654172       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c5b823d6-896c-4843-ba05-2359f195edfb","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c5b823d6-896c-4843-ba05-2359f195edfb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comr6jt5","csi.storage.k8s.io/pvc/namespace":"provisioning-8453"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:49:53.673382       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:49:53.673468       1 controllerserver.go:271] begin to create azure disk(pvc-c5b823d6-896c-4843-ba05-2359f195edfb) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:49:53.673483       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c5b823d6-896c-4843-ba05-2359f195edfb StorageAccountType:StandardSSD_LRS Size:5
I1009 02:49:56.139480       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c5b823d6-896c-4843-ba05-2359f195edfb StorageAccountType:StandardSSD_LRS Size:5
I1009 02:49:56.139524       1 controllerserver.go:348] create azure disk(pvc-c5b823d6-896c-4843-ba05-2359f195edfb) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-c5b823d6-896c-4843-ba05-2359f195edfb kubernetes.io-created-for-pvc-name:test.csi.azure.comr6jt5 kubernetes.io-created-for-pvc-namespace:provisioning-8453]) successfully
I1009 02:49:56.139549       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c5b823d6-896c-4843-ba05-2359f195edfb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comr6jt5","csi.storage.k8s.io/pvc/namespace":"provisioning-8453","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb"}}
I1009 02:50:09.174354       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 7 lines ...
I1009 02:50:56.387262       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb)
I1009 02:51:01.619924       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb
I1009 02:51:01.619955       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb) returned with <nil>
I1009 02:51:01.619974       1 utils.go:102] GRPC response: {}
I1009 02:51:07.797492       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:51:07.797520       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2","parameters":{"csi.storage.k8s.io/pv/name":"pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2","csi.storage.k8s.io/pvc/name":"test.csi.azure.com9bwpq","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5961"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:51:07.819019       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:51:07.819096       1 controllerserver.go:271] begin to create azure disk(pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:51:07.819114       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:51:10.151533       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:51:10.151607       1 controllerserver.go:348] create azure disk(pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2 kubernetes.io-created-for-pvc-name:test.csi.azure.com9bwpq kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-5961]) successfully
I1009 02:51:10.151653       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2","csi.storage.k8s.io/pvc/name":"test.csi.azure.com9bwpq","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5961","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2"}}
I1009 02:52:43.036108       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:52:43.036139       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2"}
I1009 02:52:43.036223       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2)
I1009 02:52:48.325409       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2
I1009 02:52:48.325456       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2) returned with <nil>
I1009 02:52:48.325490       1 utils.go:102] GRPC response: {}
I1009 02:52:58.481049       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:52:58.481075       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f","csi.storage.k8s.io/pvc/name":"inline-volume-tester-7zq2w-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2197"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:52:58.500053       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:52:58.500119       1 controllerserver.go:271] begin to create azure disk(pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:52:58.500135       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f StorageAccountType:StandardSSD_LRS Size:5
I1009 02:53:00.859423       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f StorageAccountType:StandardSSD_LRS Size:5
I1009 02:53:00.859477       1 controllerserver.go:348] create azure disk(pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f kubernetes.io-created-for-pvc-name:inline-volume-tester-7zq2w-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-2197]) successfully
I1009 02:53:00.859499       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f","csi.storage.k8s.io/pvc/name":"inline-volume-tester-7zq2w-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2197","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f"}}
I1009 02:54:25.730835       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:54:25.730863       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f"}
I1009 02:54:25.730943       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f)
I1009 02:54:31.074672       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f
I1009 02:54:31.074708       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f) returned with <nil>
I1009 02:54:31.074728       1 utils.go:102] GRPC response: {}
I1009 02:54:32.749924       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:54:32.749952       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-96163c7a-f6bb-4f28-ba3e-02734349125d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-96163c7a-f6bb-4f28-ba3e-02734349125d","csi.storage.k8s.io/pvc/name":"test.csi.azure.com4q586","csi.storage.k8s.io/pvc/namespace":"provisioning-2403"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:54:32.769301       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:54:32.769376       1 controllerserver.go:271] begin to create azure disk(pvc-96163c7a-f6bb-4f28-ba3e-02734349125d) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:54:32.769392       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-96163c7a-f6bb-4f28-ba3e-02734349125d StorageAccountType:StandardSSD_LRS Size:5
I1009 02:54:35.136161       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-96163c7a-f6bb-4f28-ba3e-02734349125d StorageAccountType:StandardSSD_LRS Size:5
I1009 02:54:35.136205       1 controllerserver.go:348] create azure disk(pvc-96163c7a-f6bb-4f28-ba3e-02734349125d) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-96163c7a-f6bb-4f28-ba3e-02734349125d kubernetes.io-created-for-pvc-name:test.csi.azure.com4q586 kubernetes.io-created-for-pvc-namespace:provisioning-2403]) successfully
I1009 02:54:35.136244       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-96163c7a-f6bb-4f28-ba3e-02734349125d","csi.storage.k8s.io/pvc/name":"test.csi.azure.com4q586","csi.storage.k8s.io/pvc/namespace":"provisioning-2403","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d"}}
I1009 02:55:35.159189       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:55:35.159218       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d"}
I1009 02:55:35.159306       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d)
I1009 02:55:40.462879       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d
I1009 02:55:40.462916       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d) returned with <nil>
I1009 02:55:40.462935       1 utils.go:102] GRPC response: {}
I1009 02:55:46.714729       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:55:46.714754       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360","parameters":{"csi.storage.k8s.io/pv/name":"pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360","csi.storage.k8s.io/pvc/name":"pvc-9xzqb","csi.storage.k8s.io/pvc/namespace":"provisioning-1139"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:55:46.735447       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:55:46.735518       1 controllerserver.go:271] begin to create azure disk(pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:55:46.735535       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:55:49.084029       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:55:49.084073       1 controllerserver.go:348] create azure disk(pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360 kubernetes.io-created-for-pvc-name:pvc-9xzqb kubernetes.io-created-for-pvc-namespace:provisioning-1139]) successfully
I1009 02:55:49.084098       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360","csi.storage.k8s.io/pvc/name":"pvc-9xzqb","csi.storage.k8s.io/pvc/namespace":"provisioning-1139","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360"}}
I1009 02:56:20.127072       1 utils.go:95] GRPC call: /csi.v1.Identity/GetPluginInfo
... skipping 3 lines ...
I1009 02:56:20.127693       1 utils.go:96] GRPC request: {"name":"snapshot-48926f67-ae75-4b13-a5d0-3c00bcddb676","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360"}
I1009 02:56:20.127771       1 controllerserver.go:909] begin to create snapshot(snapshot-48926f67-ae75-4b13-a5d0-3c00bcddb676, incremental: true) under rg(kubetest-enl5pco2)
I1009 02:56:22.513222       1 controllerserver.go:918] create snapshot(snapshot-48926f67-ae75-4b13-a5d0-3c00bcddb676) under rg(kubetest-enl5pco2) successfully
I1009 02:56:22.549160       1 utils.go:102] GRPC response: {"snapshot":{"creation_time":{"nanos":371815300,"seconds":1633748180},"ready_to_use":true,"size_bytes":5368709120,"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/snapshots/snapshot-48926f67-ae75-4b13-a5d0-3c00bcddb676","source_volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360"}}
I1009 02:56:24.868820       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:56:24.868884       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab","parameters":{"csi.storage.k8s.io/pv/name":"pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab","csi.storage.k8s.io/pvc/name":"pvc-hh5wq","csi.storage.k8s.io/pvc/namespace":"provisioning-1139"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}],"volume_content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/snapshots/snapshot-48926f67-ae75-4b13-a5d0-3c00bcddb676"}}}}
I1009 02:56:24.889867       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:56:24.889938       1 controllerserver.go:271] begin to create azure disk(pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:56:24.889956       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab StorageAccountType:StandardSSD_LRS Size:5
I1009 02:56:27.215721       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab StorageAccountType:StandardSSD_LRS Size:5
I1009 02:56:27.215763       1 controllerserver.go:348] create azure disk(pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab kubernetes.io-created-for-pvc-name:pvc-hh5wq kubernetes.io-created-for-pvc-namespace:provisioning-1139]) successfully
I1009 02:56:27.215788       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":{"Snapshot":{"snapshot_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/snapshots/snapshot-48926f67-ae75-4b13-a5d0-3c00bcddb676"}}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab","csi.storage.k8s.io/pvc/name":"pvc-hh5wq","csi.storage.k8s.io/pvc/namespace":"provisioning-1139","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab"}}
I1009 02:58:16.149064       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 15 lines ...
I1009 02:58:31.805591       1 utils.go:102] GRPC response: {}
I1009 02:58:33.844643       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:58:33.844669       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f"}
I1009 02:58:33.844757       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f)
I1009 02:58:34.754766       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:58:34.754792       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-9538ceb7-32b1-4297-b2df-dfc68992b508","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9538ceb7-32b1-4297-b2df-dfc68992b508","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcv8k8","csi.storage.k8s.io/pvc/namespace":"provisioning-4773"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:58:34.774537       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:58:34.774607       1 controllerserver.go:271] begin to create azure disk(pvc-9538ceb7-32b1-4297-b2df-dfc68992b508) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:58:34.774631       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9538ceb7-32b1-4297-b2df-dfc68992b508 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:58:37.157900       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9538ceb7-32b1-4297-b2df-dfc68992b508 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:58:37.157945       1 controllerserver.go:348] create azure disk(pvc-9538ceb7-32b1-4297-b2df-dfc68992b508) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-9538ceb7-32b1-4297-b2df-dfc68992b508 kubernetes.io-created-for-pvc-name:test.csi.azure.comcv8k8 kubernetes.io-created-for-pvc-namespace:provisioning-4773]) successfully
I1009 02:58:37.157969       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9538ceb7-32b1-4297-b2df-dfc68992b508","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcv8k8","csi.storage.k8s.io/pvc/namespace":"provisioning-4773","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508"}}
I1009 02:58:39.039152       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f
I1009 02:58:39.039191       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f) returned with <nil>
I1009 02:58:39.039210       1 utils.go:102] GRPC response: {}
I1009 02:58:49.390023       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:58:49.390049       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499","parameters":{"csi.storage.k8s.io/pv/name":"pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499","csi.storage.k8s.io/pvc/name":"test.csi.azure.comvnf9x","csi.storage.k8s.io/pvc/namespace":"volume-expand-1286"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:58:49.408953       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:58:49.409027       1 controllerserver.go:271] begin to create azure disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:58:49.409044       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:58:51.712897       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:58:51.712942       1 controllerserver.go:348] create azure disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499 kubernetes.io-created-for-pvc-name:test.csi.azure.comvnf9x kubernetes.io-created-for-pvc-namespace:volume-expand-1286]) successfully
I1009 02:58:51.712966       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499","csi.storage.k8s.io/pvc/name":"test.csi.azure.comvnf9x","csi.storage.k8s.io/pvc/namespace":"volume-expand-1286","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}}
I1009 02:59:34.547538       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 02:59:34.547566       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508"}
I1009 02:59:34.547640       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508)
I1009 02:59:39.796259       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508
I1009 02:59:39.796289       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508) returned with <nil>
I1009 02:59:39.796310       1 utils.go:102] GRPC response: {}
I1009 02:59:45.942381       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:59:45.942410       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226","csi.storage.k8s.io/pvc/name":"test.csi.azure.comccslg","csi.storage.k8s.io/pvc/namespace":"provisioning-3343"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 02:59:45.960817       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:59:45.960904       1 controllerserver.go:271] begin to create azure disk(pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 02:59:45.960932       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:59:48.333168       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226 StorageAccountType:StandardSSD_LRS Size:5
I1009 02:59:48.333211       1 controllerserver.go:348] create azure disk(pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226 kubernetes.io-created-for-pvc-name:test.csi.azure.comccslg kubernetes.io-created-for-pvc-namespace:provisioning-3343]) successfully
I1009 02:59:48.333235       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226","csi.storage.k8s.io/pvc/name":"test.csi.azure.comccslg","csi.storage.k8s.io/pvc/namespace":"provisioning-3343","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226"}}
I1009 03:00:04.943735       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 03:00:04.943769       1 utils.go:96] GRPC request: {}
I1009 03:00:04.943816       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 03:00:04.944436       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 03:00:04.944454       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:05.004558       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6442450944)
I1009 03:00:05.028648       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 03:00:05.028725       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 03:00:06.066707       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 03:00:06.066736       1 utils.go:96] GRPC request: {}
I1009 03:00:06.066781       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 03:00:06.067340       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 03:00:06.067359       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:06.111762       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6442450944)
I1009 03:00:06.133665       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 03:00:06.133715       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 03:00:08.146320       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 03:00:08.146348       1 utils.go:96] GRPC request: {}
I1009 03:00:08.146393       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 03:00:08.147068       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 03:00:08.147086       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:08.171491       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6442450944)
I1009 03:00:08.194053       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 03:00:08.194107       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 03:00:12.206627       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 03:00:12.206659       1 utils.go:96] GRPC request: {}
I1009 03:00:12.206702       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 03:00:12.207238       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 03:00:12.207259       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:12.229923       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6442450944)
I1009 03:00:12.251798       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 03:00:12.251842       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 03:00:20.263599       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 03:00:20.263622       1 utils.go:96] GRPC request: {}
I1009 03:00:20.263670       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 03:00:20.264229       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 03:00:20.264247       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:20.292861       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6442450944)
I1009 03:00:20.317452       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 03:00:20.317506       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 03:00:20.322377       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 03:00:20.322397       1 utils.go:96] GRPC request: {}
I1009 03:00:20.322438       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
I1009 03:00:20.322945       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerExpandVolume
I1009 03:00:20.322964       1 utils.go:96] GRPC request: {"capacity_range":{"required_bytes":6442450944},"volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:20.355907       1 controllerserver.go:807] begin to expand azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6442450944)
I1009 03:00:20.388031       1 azure_managedDiskController.go:336] azureDisk - begin to resize disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with new size(6), old size({{5 0} {<nil>}  BinarySI})
E1009 03:00:20.388084       1 utils.go:100] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) with error(azureDisk - disk resize is only supported on Unattached disk, current disk state: Attached, already attached to /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/k8s-agentpool1-31084797-vmss_0)
I1009 03:00:33.086525       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 03:00:33.086569       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5"}
I1009 03:00:33.086724       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5)
I1009 03:00:36.325334       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1009 03:00:36.325366       1 utils.go:96] GRPC request: {}
I1009 03:00:36.325423       1 utils.go:102] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":3}}},{"Type":{"Rpc":{"type":10}}},{"Type":{"Rpc":{"type":13}}}]}
... skipping 21 lines ...
I1009 03:01:48.868039       1 utils.go:102] GRPC response: {}
I1009 03:01:50.712199       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 03:01:50.712232       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d"}
I1009 03:01:50.712316       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d)
I1009 03:01:54.063590       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:01:54.063612       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlbz57","csi.storage.k8s.io/pvc/namespace":"provisioning-9114"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:01:54.084898       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:01:54.084971       1 controllerserver.go:271] begin to create azure disk(pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:01:54.084987       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:01:55.891666       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d
I1009 03:01:55.891698       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d) returned with <nil>
I1009 03:01:55.891714       1 utils.go:102] GRPC response: {}
I1009 03:01:56.477112       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:01:56.477153       1 controllerserver.go:348] create azure disk(pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3 kubernetes.io-created-for-pvc-name:test.csi.azure.comlbz57 kubernetes.io-created-for-pvc-namespace:provisioning-9114]) successfully
I1009 03:01:56.477177       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlbz57","csi.storage.k8s.io/pvc/namespace":"provisioning-9114","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3"}}
I1009 03:01:59.868042       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:01:59.868070       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-4df82415-befc-4e55-bab9-b5ba96c29e46","parameters":{"csi.storage.k8s.io/pv/name":"pvc-4df82415-befc-4e55-bab9-b5ba96c29e46","csi.storage.k8s.io/pvc/name":"test.csi.azure.combxkkq","csi.storage.k8s.io/pvc/namespace":"volume-expand-2828"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1009 03:01:59.887530       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-4df82415-befc-4e55-bab9-b5ba96c29e46, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-4df82415-befc-4e55-bab9-b5ba96c29e46' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:01:59.887638       1 controllerserver.go:271] begin to create azure disk(pvc-4df82415-befc-4e55-bab9-b5ba96c29e46) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:01:59.887657       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-4df82415-befc-4e55-bab9-b5ba96c29e46 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:02:02.306591       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-4df82415-befc-4e55-bab9-b5ba96c29e46 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:02:02.306657       1 controllerserver.go:348] create azure disk(pvc-4df82415-befc-4e55-bab9-b5ba96c29e46) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-4df82415-befc-4e55-bab9-b5ba96c29e46 kubernetes.io-created-for-pvc-name:test.csi.azure.combxkkq kubernetes.io-created-for-pvc-namespace:volume-expand-2828]) successfully
I1009 03:02:02.306682       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-4df82415-befc-4e55-bab9-b5ba96c29e46","csi.storage.k8s.io/pvc/name":"test.csi.azure.combxkkq","csi.storage.k8s.io/pvc/namespace":"volume-expand-2828","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-4df82415-befc-4e55-bab9-b5ba96c29e46"}}
I1009 03:02:05.624195       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 15 lines ...
I1009 03:03:15.937742       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3) returned with <nil>
I1009 03:03:15.937760       1 utils.go:102] GRPC response: {}
I1009 03:03:23.866385       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:03:23.866412       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c3497036-aca4-4f6f-b472-cef20212dfb8","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c3497036-aca4-4f6f-b472-cef20212dfb8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qz9qj-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2984"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:03:23.870870       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:03:23.870892       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qz9qj-my-volume-1","csi.storage.k8s.io/pvc/namespace":"ephemeral-2984"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:03:23.888869       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:03:23.888935       1 controllerserver.go:271] begin to create azure disk(pvc-c3497036-aca4-4f6f-b472-cef20212dfb8) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:03:23.888951       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c3497036-aca4-4f6f-b472-cef20212dfb8 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:03:23.999952       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:03:24.000035       1 controllerserver.go:271] begin to create azure disk(pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:03:24.000054       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:03:26.215259       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c3497036-aca4-4f6f-b472-cef20212dfb8 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:03:26.215301       1 controllerserver.go:348] create azure disk(pvc-c3497036-aca4-4f6f-b472-cef20212dfb8) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-c3497036-aca4-4f6f-b472-cef20212dfb8 kubernetes.io-created-for-pvc-name:inline-volume-tester-qz9qj-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-2984]) successfully
I1009 03:03:26.215326       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c3497036-aca4-4f6f-b472-cef20212dfb8","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qz9qj-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-2984","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8"}}
I1009 03:03:26.483613       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5 StorageAccountType:StandardSSD_LRS Size:5
... skipping 16 lines ...
I1009 03:06:42.464335       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5)
I1009 03:06:47.679899       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5
I1009 03:06:47.679930       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5) returned with <nil>
I1009 03:06:47.679946       1 utils.go:102] GRPC response: {}
I1009 03:06:50.463266       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:06:50.463292       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-e90ee62f-f04e-4958-a67e-099082dfd89a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-e90ee62f-f04e-4958-a67e-099082dfd89a","csi.storage.k8s.io/pvc/name":"test.csi.azure.com7s9mk","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-4636"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:06:50.481857       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:06:50.481927       1 controllerserver.go:271] begin to create azure disk(pvc-e90ee62f-f04e-4958-a67e-099082dfd89a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:06:50.481946       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-e90ee62f-f04e-4958-a67e-099082dfd89a StorageAccountType:StandardSSD_LRS Size:5
I1009 03:06:52.952210       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-e90ee62f-f04e-4958-a67e-099082dfd89a StorageAccountType:StandardSSD_LRS Size:5
I1009 03:06:52.952268       1 controllerserver.go:348] create azure disk(pvc-e90ee62f-f04e-4958-a67e-099082dfd89a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-e90ee62f-f04e-4958-a67e-099082dfd89a kubernetes.io-created-for-pvc-name:test.csi.azure.com7s9mk kubernetes.io-created-for-pvc-namespace:fsgroupchangepolicy-4636]) successfully
I1009 03:06:52.952307       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e90ee62f-f04e-4958-a67e-099082dfd89a","csi.storage.k8s.io/pvc/name":"test.csi.azure.com7s9mk","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-4636","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a"}}
I1009 03:07:04.241198       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 187 lines ...
I1009 03:07:17.691559       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0c30bc9f-852c-48a5-8564-a4a33da18284)
I1009 03:07:22.975326       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0c30bc9f-852c-48a5-8564-a4a33da18284
I1009 03:07:22.975357       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0c30bc9f-852c-48a5-8564-a4a33da18284) returned with <nil>
I1009 03:07:22.975373       1 utils.go:102] GRPC response: {}
I1009 03:07:29.422155       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:07:29.422189       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078","parameters":{"csi.storage.k8s.io/pv/name":"pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlznpd","csi.storage.k8s.io/pvc/namespace":"volume-expand-6092"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:07:29.440016       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:07:29.440094       1 controllerserver.go:271] begin to create azure disk(pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:07:29.440112       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:07:31.951424       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078 StorageAccountType:StandardSSD_LRS Size:5
I1009 03:07:31.951466       1 controllerserver.go:348] create azure disk(pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078 kubernetes.io-created-for-pvc-name:test.csi.azure.comlznpd kubernetes.io-created-for-pvc-namespace:volume-expand-6092]) successfully
I1009 03:07:31.951490       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlznpd","csi.storage.k8s.io/pvc/namespace":"volume-expand-6092","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-50ee5f57-8221-4a00-90e3-8c6cd765f078"}}
I1009 03:08:04.778631       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 7 lines ...
I1009 03:08:11.736390       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a)
I1009 03:08:16.952865       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a
I1009 03:08:16.952896       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a) returned with <nil>
I1009 03:08:16.952914       1 utils.go:102] GRPC response: {}
I1009 03:08:23.531687       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:08:23.531729       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea","parameters":{"csi.storage.k8s.io/pv/name":"pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea","csi.storage.k8s.io/pvc/name":"inline-volume-tester-nzkwr-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5197"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:08:23.570500       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:08:23.570611       1 controllerserver.go:271] begin to create azure disk(pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:08:23.570627       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea StorageAccountType:StandardSSD_LRS Size:5
I1009 03:08:25.888047       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea StorageAccountType:StandardSSD_LRS Size:5
I1009 03:08:25.888108       1 controllerserver.go:348] create azure disk(pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea kubernetes.io-created-for-pvc-name:inline-volume-tester-nzkwr-my-volume-0 kubernetes.io-created-for-pvc-namespace:ephemeral-5197]) successfully
I1009 03:08:25.888131       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea","csi.storage.k8s.io/pvc/name":"inline-volume-tester-nzkwr-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5197","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea"}}
I1009 03:09:51.582331       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 03:09:51.582380       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea"}
I1009 03:09:51.582482       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea)
I1009 03:09:56.849068       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea
I1009 03:09:56.849120       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea) returned with <nil>
I1009 03:09:56.849138       1 utils.go:102] GRPC response: {}
I1009 03:09:58.752852       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:09:58.752879       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":5368709120},"name":"pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c","csi.storage.k8s.io/pvc/name":"test.csi.azure.commrjg7","csi.storage.k8s.io/pvc/namespace":"volume-8542"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":7}}]}
I1009 03:09:58.771859       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:09:58.771965       1 controllerserver.go:271] begin to create azure disk(pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) diskZone() maxShares(0)
I1009 03:09:58.771987       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c StorageAccountType:StandardSSD_LRS Size:5
I1009 03:10:01.134101       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c StorageAccountType:StandardSSD_LRS Size:5
I1009 03:10:01.134145       1 controllerserver.go:348] create azure disk(pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(5) tags(map[kubernetes.io-created-for-pv-name:pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c kubernetes.io-created-for-pvc-name:test.csi.azure.commrjg7 kubernetes.io-created-for-pvc-namespace:volume-8542]) successfully
I1009 03:10:01.134170       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":5368709120,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c","csi.storage.k8s.io/pvc/name":"test.csi.azure.commrjg7","csi.storage.k8s.io/pvc/namespace":"volume-8542","requestedsizegib":"5"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c"}}
I1009 03:11:23.922261       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
I1009 03:11:23.922291       1 utils.go:96] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c"}
I1009 03:11:23.922391       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c)
I1009 03:11:29.120812       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c
I1009 03:11:29.120860       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c040cd90-913c-4d48-8189-8b46f3d7a93c) returned with <nil>
I1009 03:11:29.120878       1 utils.go:102] GRPC response: {}
I1009 03:11:34.237703       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:11:34.237730       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-35e19790-8c6b-4b17-be86-67c5a817f35a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-35e19790-8c6b-4b17-be86-67c5a817f35a","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:11:34.259991       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-35e19790-8c6b-4b17-be86-67c5a817f35a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-35e19790-8c6b-4b17-be86-67c5a817f35a' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:11:34.260081       1 controllerserver.go:271] begin to create azure disk(pvc-35e19790-8c6b-4b17-be86-67c5a817f35a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) diskZone() maxShares(0)
I1009 03:11:34.260099       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-35e19790-8c6b-4b17-be86-67c5a817f35a StorageAccountType:StandardSSD_LRS Size:10
I1009 03:11:36.615690       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-35e19790-8c6b-4b17-be86-67c5a817f35a StorageAccountType:StandardSSD_LRS Size:10
I1009 03:11:36.615733       1 controllerserver.go:348] create azure disk(pvc-35e19790-8c6b-4b17-be86-67c5a817f35a) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-35e19790-8c6b-4b17-be86-67c5a817f35a kubernetes.io-created-for-pvc-name:pvc-azuredisk kubernetes.io-created-for-pvc-namespace:default]) successfully
I1009 03:11:36.615757       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-35e19790-8c6b-4b17-be86-67c5a817f35a","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-35e19790-8c6b-4b17-be86-67c5a817f35a"}}
I1009 03:12:00.920607       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:12:00.920633       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd","parameters":{"csi.storage.k8s.io/pv/name":"pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:12:00.941043       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:12:00.941132       1 controllerserver.go:271] begin to create azure disk(pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) diskZone() maxShares(0)
I1009 03:12:00.941149       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd StorageAccountType:StandardSSD_LRS Size:10
I1009 03:12:03.253702       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd StorageAccountType:StandardSSD_LRS Size:10
I1009 03:12:03.253746       1 controllerserver.go:348] create azure disk(pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd kubernetes.io-created-for-pvc-name:persistent-storage-statefulset-azuredisk-0 kubernetes.io-created-for-pvc-namespace:default]) successfully
I1009 03:12:03.253770       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-202ed9fb-f19a-4285-8cfa-de6b064573fd"}}
I1009 03:12:36.208568       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:12:36.208597       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-824e414a-478d-4253-8f19-161eb804bded","parameters":{"csi.storage.k8s.io/pv/name":"pvc-824e414a-478d-4253-8f19-161eb804bded","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:12:36.229118       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-824e414a-478d-4253-8f19-161eb804bded, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-824e414a-478d-4253-8f19-161eb804bded' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:12:36.229204       1 controllerserver.go:271] begin to create azure disk(pvc-824e414a-478d-4253-8f19-161eb804bded) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) diskZone() maxShares(0)
I1009 03:12:36.229235       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-824e414a-478d-4253-8f19-161eb804bded StorageAccountType:StandardSSD_LRS Size:10
I1009 03:12:38.805217       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-824e414a-478d-4253-8f19-161eb804bded StorageAccountType:StandardSSD_LRS Size:10
I1009 03:12:38.805292       1 controllerserver.go:348] create azure disk(pvc-824e414a-478d-4253-8f19-161eb804bded) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-824e414a-478d-4253-8f19-161eb804bded kubernetes.io-created-for-pvc-name:persistent-storage-statefulset-azuredisk-nonroot-0 kubernetes.io-created-for-pvc-namespace:default]) successfully
I1009 03:12:38.805317       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-824e414a-478d-4253-8f19-161eb804bded","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-824e414a-478d-4253-8f19-161eb804bded"}}
I1009 03:13:04.668944       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:13:04.668974       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999","parameters":{"csi.storage.k8s.io/pv/name":"pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999","csi.storage.k8s.io/pvc/name":"nginx-azuredisk-ephemeral-azuredisk01","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:13:04.688585       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:13:04.688725       1 controllerserver.go:271] begin to create azure disk(pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) diskZone() maxShares(0)
I1009 03:13:04.688753       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999 StorageAccountType:StandardSSD_LRS Size:10
I1009 03:13:07.096755       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999 StorageAccountType:StandardSSD_LRS Size:10
I1009 03:13:07.096799       1 controllerserver.go:348] create azure disk(pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999 kubernetes.io-created-for-pvc-name:nginx-azuredisk-ephemeral-azuredisk01 kubernetes.io-created-for-pvc-namespace:default]) successfully
I1009 03:13:07.096822       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999","csi.storage.k8s.io/pvc/name":"nginx-azuredisk-ephemeral-azuredisk01","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b2cadab3-f630-41b8-a04f-acd09ffb5999"}}
I1009 03:13:44.963750       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:13:44.963781       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-56e60524-6b68-41ee-b164-bb760ea3d38e","parameters":{"csi.storage.k8s.io/pv/name":"pvc-56e60524-6b68-41ee-b164-bb760ea3d38e","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-wgfwt-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:13:44.985402       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 03:13:44.985431       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.test.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da","parameters":{"csi.storage.k8s.io/pv/name":"pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-hbpt2-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]}
I1009 03:13:44.991041       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-56e60524-6b68-41ee-b164-bb760ea3d38e, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-56e60524-6b68-41ee-b164-bb760ea3d38e' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:13:44.991113       1 controllerserver.go:271] begin to create azure disk(pvc-56e60524-6b68-41ee-b164-bb760ea3d38e) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) diskZone() maxShares(0)
I1009 03:13:44.991129       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-56e60524-6b68-41ee-b164-bb760ea3d38e StorageAccountType:StandardSSD_LRS Size:10
I1009 03:13:45.023923       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da' under resource group 'kubetest-enl5pco2' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 03:13:45.024006       1 controllerserver.go:271] begin to create azure disk(pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) diskZone() maxShares(0)
I1009 03:13:45.024023       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da StorageAccountType:StandardSSD_LRS Size:10
I1009 03:13:47.384024       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da StorageAccountType:StandardSSD_LRS Size:10
I1009 03:13:47.384085       1 controllerserver.go:348] create azure disk(pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da) account type(StandardSSD_LRS) rg(kubetest-enl5pco2) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da kubernetes.io-created-for-pvc-name:daemonset-azuredisk-ephemeral-hbpt2-azuredisk kubernetes.io-created-for-pvc-namespace:default]) successfully
I1009 03:13:47.384110       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.test.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da","csi.storage.k8s.io/pvc/name":"daemonset-azuredisk-ephemeral-hbpt2-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-41ca9db9-4d9f-4f19-ba9e-cd54385b14da"}}
I1009 03:13:47.973104       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-56e60524-6b68-41ee-b164-bb760ea3d38e StorageAccountType:StandardSSD_LRS Size:10
... skipping 14 lines ...
Platform: linux/amd64
Topology Key: topology.test.csi.azure.com/zone

Streaming logs below:
I1009 02:41:23.283729       1 azuredisk.go:139] driver userAgent: test.csi.azure.com/v1.8.0-9eaf1e00dfb23da1f5430a51a79908d1683f455a gc/go1.17.1 (amd64-linux) e2e-test
I1009 02:41:23.285255       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1009 02:41:23.380630       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1009 02:41:23.380660       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1009 02:41:23.380668       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1009 02:41:23.380732       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1009 02:41:23.381441       1 azure_auth.go:234] Using AzurePublicCloud environment
I1009 02:41:23.381489       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I1009 02:41:23.381495       1 azure_auth.go:102] azure: using User Assigned MSI ID to retrieve access token
... skipping 92 lines ...
I1009 02:41:37.334764       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:37.424010       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-2eff62da-3354-4f5b-9301-00fde723174a" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:41:37.424056       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:37.424076       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-2eff62da-3354-4f5b-9301-00fde723174a" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-2eff62da-3354-4f5b-9301-00fde723174a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2eff62da-3354-4f5b-9301-00fde723174a  true false 0})]
I1009 02:41:37.424116       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:37.424130       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-2eff62da-3354-4f5b-9301-00fde723174a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2eff62da-3354-4f5b-9301-00fde723174a  true false 0})])
I1009 02:41:37.844447       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-2eff62da-3354-4f5b-9301-00fde723174a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2eff62da-3354-4f5b-9301-00fde723174a  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:41:38.172286       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:41:38.172314       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-165d8edd-c536-4637-826e-600f10d4255a","csi.storage.k8s.io/pvc/name":"test.csi.azure.comxjffk","csi.storage.k8s.io/pvc/namespace":"volumemode-4807","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a"}
I1009 02:41:38.195164       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:41:38.195199       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:38.195221       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:41:38.195239       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 2 lines ...
I1009 02:41:38.195298       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-165d8edd-c536-4637-826e-600f10d4255a  true false 0})])
I1009 02:41:38.271050       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:41:38.271078       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcxtfr","csi.storage.k8s.io/pvc/namespace":"volume-expand-3165","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb"}
I1009 02:41:38.376798       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb" to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:41:38.376836       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:38.376857       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:41:38.377525       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-165d8edd-c536-4637-826e-600f10d4255a  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:41:38.496549       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:41:38.496593       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:38.496641       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb" lun 1 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb  true false 1})]
I1009 02:41:38.496663       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:38.496678       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb  true false 1})])
I1009 02:41:38.733686       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:41:39.182444       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:41:39.182533       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f","csi.storage.k8s.io/pvc/name":"inline-volume-tester-w2csj-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f"}
I1009 02:41:39.205070       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:41:39.205124       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:39.205163       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:41:39.264756       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f" to node "k8s-agentpool1-31084797-vmss000001"
... skipping 7 lines ...
I1009 02:41:39.321817       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:39.321873       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:41:39.321887       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:39.321898       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e" lun 2 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1efada0-40c8-497a-9795-6ed4682d940e  true false 2})]
I1009 02:41:39.321930       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:39.321961       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1efada0-40c8-497a-9795-6ed4682d940e  true false 2})])
I1009 02:41:39.428924       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8bcf1dea-bf9e-43a5-afba-af5028b76c3f  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:41:39.482884       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:41:39.482912       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e","csi.storage.k8s.io/pvc/name":"test.csi.azure.compgjr8","csi.storage.k8s.io/pvc/namespace":"volumemode-8971","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e"}
I1009 02:41:39.509085       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:41:39.509122       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:39.509144       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:41:39.552463       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:41:39.552507       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:39.552527       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e" lun 2 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e  true false 2})]
I1009 02:41:39.552543       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:41:39.552556       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e  true false 2})])
I1009 02:41:39.711160       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:41:40.056589       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1efada0-40c8-497a-9795-6ed4682d940e  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:41:48.602427       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:41:48.602466       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-165d8edd-c536-4637-826e-600f10d4255a" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 02:41:48.602505       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:41:48.820608       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:41:48.820645       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:41:48.820665       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 213 lines ...
I1009 02:42:28.971124       1 utils.go:102] GRPC response: {}
I1009 02:42:28.971258       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:28.971293       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:29.035372       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" lun 30 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-15c1b3cd-19a6-4076-b236-6fb749963aca  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-21c3a0e3-938f-4782-b96c-5133045752b8  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39b3fa75-3672-44d3-80aa-5c8b181102df  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f30ec72-2180-4324-af67-1c21807af88e  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5878152e-4c5d-4ae9-9862-d45656993cae  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6df84773-5354-4790-958d-beb160084ccd  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77114e1d-ced2-44d6-b825-6e13cb80e212  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9de70f81-5326-4e06-9cba-36c71821fbe2  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6307f77-27bf-4d29-a99b-6656c918312f  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6cf353d-f05a-49be-8090-56795f944cf4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bdc2dd19-74a6-43e0-b235-d251154ece04  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a  true false 34}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e8570a05-f168-4387-bab6-c00121b30c84  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f39fe354-8111-4de5-b660-c804ed08f0de  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 29})]
I1009 02:42:29.035516       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:29.035589       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-15c1b3cd-19a6-4076-b236-6fb749963aca  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-21c3a0e3-938f-4782-b96c-5133045752b8  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39b3fa75-3672-44d3-80aa-5c8b181102df  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f30ec72-2180-4324-af67-1c21807af88e  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5878152e-4c5d-4ae9-9862-d45656993cae  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6df84773-5354-4790-958d-beb160084ccd  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77114e1d-ced2-44d6-b825-6e13cb80e212  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9de70f81-5326-4e06-9cba-36c71821fbe2  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6307f77-27bf-4d29-a99b-6656c918312f  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6cf353d-f05a-49be-8090-56795f944cf4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bdc2dd19-74a6-43e0-b235-d251154ece04  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a  true false 34}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e8570a05-f168-4387-bab6-c00121b30c84  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f39fe354-8111-4de5-b660-c804ed08f0de  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 29})])
I1009 02:42:29.115925       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:42:29.115956       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:42:29.116154       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-15c1b3cd-19a6-4076-b236-6fb749963aca  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-21c3a0e3-938f-4782-b96c-5133045752b8  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39b3fa75-3672-44d3-80aa-5c8b181102df  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f30ec72-2180-4324-af67-1c21807af88e  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5878152e-4c5d-4ae9-9862-d45656993cae  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6df84773-5354-4790-958d-beb160084ccd  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77114e1d-ced2-44d6-b825-6e13cb80e212  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9de70f81-5326-4e06-9cba-36c71821fbe2  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6307f77-27bf-4d29-a99b-6656c918312f  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6cf353d-f05a-49be-8090-56795f944cf4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bdc2dd19-74a6-43e0-b235-d251154ece04  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a  true false 34}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e8570a05-f168-4387-bab6-c00121b30c84  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f39fe354-8111-4de5-b660-c804ed08f0de  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 29})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:42:29.116304       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-15c1b3cd-19a6-4076-b236-6fb749963aca  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-21c3a0e3-938f-4782-b96c-5133045752b8  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39b3fa75-3672-44d3-80aa-5c8b181102df  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f30ec72-2180-4324-af67-1c21807af88e  true false 27}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5878152e-4c5d-4ae9-9862-d45656993cae  true false 23}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6df84773-5354-4790-958d-beb160084ccd  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77114e1d-ced2-44d6-b825-6e13cb80e212  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a  true false 22}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9de70f81-5326-4e06-9cba-36c71821fbe2  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 24}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38  true false 26}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7  true false 3}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6307f77-27bf-4d29-a99b-6656c918312f  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6cf353d-f05a-49be-8090-56795f944cf4  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e  true false 28}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bdc2dd19-74a6-43e0-b235-d251154ece04  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4  true false 18}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a  true false 34}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734  true false 8}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 21}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e8570a05-f168-4387-bab6-c00121b30c84  true false 31}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f39fe354-8111-4de5-b660-c804ed08f0de  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 29})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:42:29.116399       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:42:29.116459       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:42:29.116471       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:29.116497       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:29.121899       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:29.121920       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df","csi.storage.k8s.io/pvc/name":"pvc-s87rd","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df"}
I1009 02:42:29.145402       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:29.145451       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159131       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159160       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" to node "k8s-agentpool1-31084797-vmss000001"
E1009 02:42:29.159173       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159184       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159257       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159294       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159315       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159337       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153) in current disk list(len: 2) nor in diskMap(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df:0xc00004a740])
E1009 02:42:29.159355       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153) in current disk list(len: 2) nor in diskMap(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df:0xc00004a740])
I1009 02:42:29.159392       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159406       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159428       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159443       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159486       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159516       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159550       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159568       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159590       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159622       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159674       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159701       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159735       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159753       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159794       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159821       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159848       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159870       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159892       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159916       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159933       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159946       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.159960       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.159974       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.159994       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160009       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160025       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160026       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160056       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160093       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160124       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160143       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160160       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160173       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160188       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160206       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160221       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160234       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160251       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160261       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160275       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160302       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160313       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160328       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160343       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160354       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160373       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160388       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160407       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160446       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160468       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160496       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160513       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160530       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160543       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160557       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160572       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160584       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160598       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160614       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160634       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160647       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160669       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160683       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160711       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160731       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160742       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160761       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160780       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160809       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160830       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160848       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160889       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160909       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.160933       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.160945       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.160966       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.161000       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.161014       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.161036       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.161055       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.161066       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.161079       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.161094       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.161105       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.161146       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.161164       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.161175       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:29.161188       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df) in current disk list(len: 2) nor in diskMap(map[])
E1009 02:42:29.161203       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df) in current disk list(len: 2) nor in diskMap(map[])
I1009 02:42:29.276289       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:29.276328       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7","csi.storage.k8s.io/pvc/name":"pvc-kpdqs","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7"}
I1009 02:42:29.277104       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:29.277125       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2","csi.storage.k8s.io/pvc/name":"pvc-9h4sb","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2"}
I1009 02:42:29.300520       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:29.300574       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 2 lines ...
I1009 02:42:29.300644       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2" lun 0 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2  true false 0})]
I1009 02:42:29.300662       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:29.300680       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2  true false 0})])
I1009 02:42:29.305447       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:29.305474       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:29.305494       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:29.450364       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9cedd078-0274-4ef4-a598-d5d635de7aa2  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:29.450543       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:29.450575       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:29.476232       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:29.476261       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153","csi.storage.k8s.io/pvc/name":"pvc-zjcmw","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153"}
I1009 02:42:29.476770       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:29.476791       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e","csi.storage.k8s.io/pvc/name":"pvc-bqbw4","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e"}
... skipping 74 lines ...
I1009 02:42:30.625407       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:30.778355       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:30.778385       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b6cf353d-f05a-49be-8090-56795f944cf4","csi.storage.k8s.io/pvc/name":"pvc-h54xk","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4"}
I1009 02:42:30.803256       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:30.803289       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:30.803316       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:30.952438       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b30d40d9-f7c7-4102-8fd5-6312cbd811e7  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:30.952620       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:30.952669       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:30.978712       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:30.978742       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431","csi.storage.k8s.io/pvc/name":"pvc-8f7jm","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431"}
I1009 02:42:31.000542       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:31.000579       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 8 lines ...
I1009 02:42:31.334599       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:31.380109       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:31.380145       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-39b3fa75-3672-44d3-80aa-5c8b181102df","csi.storage.k8s.io/pvc/name":"pvc-hk7wr","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df"}
I1009 02:42:31.408326       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:31.408382       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.408412       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:31.495577       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153  true false 12}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-3f30ec72-2180-4324-af67-1c21807af88e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3f30ec72-2180-4324-af67-1c21807af88e  true false 15}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df  true false 4}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6df84773-5354-4790-958d-beb160084ccd:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6df84773-5354-4790-958d-beb160084ccd  true false 16}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-77114e1d-ced2-44d6-b825-6e13cb80e212  true false 6}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d  true false 14}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38  true false 7}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6307f77-27bf-4d29-a99b-6656c918312f  true false 11}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b6cf353d-f05a-49be-8090-56795f944cf4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b6cf353d-f05a-49be-8090-56795f944cf4  true false 17}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e  true false 13}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-bdc2dd19-74a6-43e0-b235-d251154ece04  true false 10}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7  true false 5}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e8570a05-f168-4387-bab6-c00121b30c84  true false 9}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f39fe354-8111-4de5-b660-c804ed08f0de  true false 8})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:31.495745       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.495820       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:31.542173       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-3302a299-56be-4b35-9bfe-a3ddc7b94153" lun 12 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39b3fa75-3672-44d3-80aa-5c8b181102df  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431  true false 18})]
I1009 02:42:31.542249       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.542308       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39b3fa75-3672-44d3-80aa-5c8b181102df  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431  true false 18})])
I1009 02:42:31.577310       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:31.577338       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c","csi.storage.k8s.io/pvc/name":"pvc-2fs5n","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c"}
I1009 02:42:31.599620       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:31.599659       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.599695       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:31.740964       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-39b3fa75-3672-44d3-80aa-5c8b181102df:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-39b3fa75-3672-44d3-80aa-5c8b181102df  true false 20}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7424cd6a-2837-4b7a-81ad-41c0335ed198  true false 19}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-78ef4107-0c5e-4d8a-bb87-c503697d4431  true false 18})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:31.741126       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.741167       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:31.777442       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:31.777468       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a","csi.storage.k8s.io/pvc/name":"pvc-r8z8x","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a"}
I1009 02:42:31.787839       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212" lun 6 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c  true false 21})]
I1009 02:42:31.787881       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.787915       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c  true false 21})])
I1009 02:42:31.805707       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:31.805739       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.805803       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:31.951611       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c  true false 21})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:31.951747       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:31.951798       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:31.978449       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:31.978477       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a","csi.storage.k8s.io/pvc/name":"pvc-66t68","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a"}
I1009 02:42:32.001734       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:32.001800       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.005290       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e" lun 13 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a  true false 22})]
I1009 02:42:32.005332       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.005366       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a  true false 22})])
I1009 02:42:32.005365       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:32.170324       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-8a73bc68-6a7b-46d4-99a7-10798db9d43a  true false 22})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:32.170477       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.170511       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:32.177623       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:32.177644       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-21c3a0e3-938f-4782-b96c-5133045752b8","csi.storage.k8s.io/pvc/name":"pvc-jlxpl","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8"}
I1009 02:42:32.201065       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:32.201097       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.218074       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-ad5bcbcf-1fbf-42c6-bb58-07bc6eafca38" lun 7 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a  true false 23})]
I1009 02:42:32.218106       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.218160       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a  true false 23})])
I1009 02:42:32.218288       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:32.377686       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:32.377716       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-15c1b3cd-19a6-4076-b236-6fb749963aca","csi.storage.k8s.io/pvc/name":"pvc-t2kjm","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca"}
I1009 02:42:32.396215       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5fc2e5a-f336-4521-8a0d-96e40fd7d50a  true false 23})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:32.396352       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.396379       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:32.401460       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:32.401490       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.494926       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1ed90d8-53b1-4c51-ab71-7d882b8cb51d" lun 14 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-21c3a0e3-938f-4782-b96c-5133045752b8  true false 24})]
I1009 02:42:32.494975       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.494968       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:32.495013       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-21c3a0e3-938f-4782-b96c-5133045752b8  true false 24})])
I1009 02:42:32.577924       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:32.577953       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9de70f81-5326-4e06-9cba-36c71821fbe2","csi.storage.k8s.io/pvc/name":"pvc-v8nzz","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2"}
I1009 02:42:32.601625       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:32.601662       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.601699       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:32.649494       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-21c3a0e3-938f-4782-b96c-5133045752b8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-21c3a0e3-938f-4782-b96c-5133045752b8  true false 24})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:32.649685       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.649711       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:32.713334       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e8570a05-f168-4387-bab6-c00121b30c84" lun 9 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-15c1b3cd-19a6-4076-b236-6fb749963aca  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9de70f81-5326-4e06-9cba-36c71821fbe2  true false 26})]
I1009 02:42:32.713385       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.713440       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-15c1b3cd-19a6-4076-b236-6fb749963aca  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9de70f81-5326-4e06-9cba-36c71821fbe2  true false 26})])
I1009 02:42:32.777741       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:32.777775       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-5878152e-4c5d-4ae9-9862-d45656993cae","csi.storage.k8s.io/pvc/name":"pvc-lhtw6","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae"}
I1009 02:42:32.800411       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:32.800448       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.800484       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:32.917049       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-15c1b3cd-19a6-4076-b236-6fb749963aca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-15c1b3cd-19a6-4076-b236-6fb749963aca  true false 25}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9de70f81-5326-4e06-9cba-36c71821fbe2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9de70f81-5326-4e06-9cba-36c71821fbe2  true false 26})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:32.917335       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.917368       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:32.977003       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:32.977035       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4","csi.storage.k8s.io/pvc/name":"pvc-t7nhc","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4"}
I1009 02:42:32.979952       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f39fe354-8111-4de5-b660-c804ed08f0de" lun 8 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5878152e-4c5d-4ae9-9862-d45656993cae  true false 27})]
I1009 02:42:32.979991       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:32.980037       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5878152e-4c5d-4ae9-9862-d45656993cae  true false 27})])
I1009 02:42:33.005126       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:33.005180       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.005254       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:33.135279       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-5878152e-4c5d-4ae9-9862-d45656993cae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-5878152e-4c5d-4ae9-9862-d45656993cae  true false 27})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:33.135429       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.135467       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:33.178311       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:33.178339       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734","csi.storage.k8s.io/pvc/name":"pvc-4pp69","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734"}
I1009 02:42:33.200642       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d60d5f3d-29ab-45f8-9140-6b5ac59572c7" lun 5 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4  true false 28})]
I1009 02:42:33.200694       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.200751       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4  true false 28})])
I1009 02:42:33.212598       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:33.212639       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.212678       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:33.378139       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:33.378167       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09","csi.storage.k8s.io/pvc/name":"pvc-lmlfs","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09"}
I1009 02:42:33.384727       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c068b98e-4b0b-4bc2-a2b8-e3a0062609d4  true false 28})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:33.384871       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.384896       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:33.403154       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:33.403193       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.430264       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6313e55b-5970-4a06-99ec-bdc92f59c8df" lun 4 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734  true false 29})]
I1009 02:42:33.430325       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 6 lines ...
I1009 02:42:33.600039       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:33.777414       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:33.777444       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-aaec181d-b96b-4280-9361-eab52a2a07d5","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-jmvp4-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5"}
I1009 02:42:33.804265       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:33.804302       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.804341       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:42:33.819637       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734  true false 29})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:33.819812       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.819840       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:33.881071       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04" lun 10 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 31})]
I1009 02:42:33.881123       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:33.881206       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 31})])
I1009 02:42:33.969350       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:42:33.969382       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:42:33.969435       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 31})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:42:33.969479       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 30}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 31})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:42:33.969542       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:42:33.969575       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bdc2dd19-74a6-43e0-b235-d251154ece04" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:42:33.969644       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 10 lines ...
I1009 02:42:34.032454       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.032475       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2" lun 30 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2  true false 30})]
I1009 02:42:34.032495       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.032531       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2  true false 30})])
I1009 02:42:34.180047       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:42:34.180075       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24","csi.storage.k8s.io/pvc/name":"pvc-4x299","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24"}
I1009 02:42:34.204097       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-44007c6e-06a2-45d6-9172-53fbf9cba9f2  true false 30})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:34.204260       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.204286       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:34.208117       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:34.208145       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.272676       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b6307f77-27bf-4d29-a99b-6656c918312f" lun 11 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[]
I1009 02:42:34.272755       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24" to node "k8s-agentpool1-31084797-vmss000001"
... skipping 23 lines ...
I1009 02:42:34.377483       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e","csi.storage.k8s.io/pvc/name":"test.csi.azure.compgjr8","csi.storage.k8s.io/pvc/namespace":"volumemode-8971","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e"}
I1009 02:42:34.400974       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:34.401013       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.401038       1 azure_controller_common.go:457] azureDisk - find disk: lun 2 name "pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e"
I1009 02:42:34.401050       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-efe86b8a-6aa0-498f-8250-b330a9b7c89e" is already attached to node "k8s-agentpool1-31084797-vmss000001" at lun 2.
I1009 02:42:34.401086       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I1009 02:42:34.494562       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24  true false 31})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:42:34.494700       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.494739       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:42:34.547385       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c" lun 21 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[]
I1009 02:42:34.547442       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:34.547456       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-8b13178b-2ae4-4b5e-a7e7-a7e57ceb102c" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 02:42:34.547483       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 35 lines ...
I1009 02:42:34.548487       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.548510       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734" lun 29 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[]
I1009 02:42:34.548525       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:34.548535       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3d4500d-f5fc-4076-8ee1-b3595afe4734" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 02:42:34.548554       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"29"}}
I1009 02:42:34.548577       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:34.548621       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09) in current disk list(len: 32) nor in diskMap(map[])
I1009 02:42:34.548640       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:34.548647       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09) in current disk list(len: 32) nor in diskMap(map[])
E1009 02:42:34.548678       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 32) nor in diskMap(map[])
E1009 02:42:34.548701       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 32) nor in diskMap(map[])
I1009 02:42:34.548728       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:42:34.548759       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 32) nor in diskMap(map[])
E1009 02:42:34.548779       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 32) nor in diskMap(map[])
I1009 02:42:34.548794       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:42:34.548822       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24" lun 31 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[]
I1009 02:42:34.548834       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:42:34.548844       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-eb8a2350-0558-4d54-b4f8-3063dd87ab24" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 02:42:34.548856       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"31"}}
I1009 02:42:34.784441       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 141 lines ...
I1009 02:43:21.600394       1 utils.go:102] GRPC response: {}
I1009 02:43:21.600453       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.600469       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:43:21.648379       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 32})]
I1009 02:43:21.648444       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.648512       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 32})])
I1009 02:43:21.721886       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:21.721919       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:43:21.721973       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:43:21.722020       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:43:21.722079       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:43:21.722110       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:21.722196       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.722232       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:43:21.727460       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:43:21.727484       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09","csi.storage.k8s.io/pvc/name":"pvc-lmlfs","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09"}
I1009 02:43:21.767802       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:43:21.767842       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:43:21.772242       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 31) nor in diskMap(map[])
E1009 02:43:21.772291       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 31) nor in diskMap(map[])
I1009 02:43:21.772329       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:43:21.772354       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:43:21.772399       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 31) nor in diskMap(map[])
E1009 02:43:21.772423       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 31) nor in diskMap(map[])
I1009 02:43:21.772454       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.772476       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09" lun 2 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 2})]
I1009 02:43:21.772492       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.772525       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 2})])
I1009 02:43:21.778081       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:43:21.778114       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-aaec181d-b96b-4280-9361-eab52a2a07d5","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-jmvp4-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5"}
... skipping 2 lines ...
I1009 02:43:21.800487       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:43:21.800527       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.800572       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:43:21.801865       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:43:21.801906       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.801986       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:43:21.982944       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-fc5f07fa-401c-4cfc-ae39-2395e76caf09  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:43:21.983121       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:21.983143       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:43:22.034640       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 33})]
I1009 02:43:22.034689       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:22.034743       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 33})])
I1009 02:43:22.093318       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:22.093363       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:43:22.093419       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 33})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:43:22.093464       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 33})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:43:22.093516       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:43:22.093547       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:22.093591       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:22.093618       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
E1009 02:43:22.144650       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 32) nor in diskMap(map[])
E1009 02:43:22.144700       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 32) nor in diskMap(map[])
I1009 02:43:22.203022       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:43:22.203067       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-77114e1d-ced2-44d6-b825-6e13cb80e212" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 02:43:22.203087       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"6"}}
I1009 02:43:22.544695       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:43:22.544735       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-bc55774b-676e-4d9c-8dc5-821e3d8b428e" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 02:43:22.544754       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"13"}}
... skipping 11 lines ...
I1009 02:43:25.804104       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]
I1009 02:43:25.804154       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:25.804214       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})])
I1009 02:43:25.807680       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:43:25.807710       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:25.807744       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:43:25.870299       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:25.870344       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:43:25.870420       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:43:25.870447       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:43:25.870550       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:43:25.870615       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:25.870665       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:25.870724       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:43:25.936380       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 02:43:25.936421       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:25.936476       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 02:43:26.011585       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:26.011619       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:43:26.011673       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:43:26.011715       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:43:26.011758       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:43:26.011811       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:26.727171       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e:pvc-a1efada0-40c8-497a-9795-6ed4682d940e]) returned with <nil>
... skipping 7 lines ...
I1009 02:43:26.781408       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-a1efada0-40c8-497a-9795-6ed4682d940e from node k8s-agentpool1-31084797-vmss000000 successfully
I1009 02:43:26.781458       1 utils.go:102] GRPC response: {}
I1009 02:43:26.781526       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:26.781573       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47  true false 1})]
I1009 02:43:26.781606       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:26.781622       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47  true false 1})])
I1009 02:43:26.952592       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6  true false 0}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:43:26.952853       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:26.952889       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:43:27.003427       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47" lun 1 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[]
I1009 02:43:27.003475       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:43:27.003490       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:43:27.003508       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 3 lines ...
I1009 02:43:33.622617       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:33.622642       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:43:33.622660       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:33.622673       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb" lun 2 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb  true false 2})]
I1009 02:43:33.622690       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:33.622707       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb  true false 2})])
I1009 02:43:33.788532       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9898f58a-a8d2-4710-adb6-6731b82b8deb  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:43:41.882496       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:43:41.882524       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-aaec181d-b96b-4280-9361-eab52a2a07d5","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-jmvp4-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5"}
I1009 02:43:41.904687       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:43:41.904716       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:41.904774       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:43:41.904806       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:41.904863       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]
I1009 02:43:41.904898       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:41.904929       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})])
I1009 02:43:41.983350       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:41.983382       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:43:41.983433       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:43:41.983468       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:43:41.983504       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:43:41.983537       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:42.019683       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:43:42.041615       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:43:42.106542       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:43:42.106587       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:42.106627       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 02:43:42.106645       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:43:42.106677       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 02:43:42.163488       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:42.163518       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:43:42.163568       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:43:42.163599       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:43:42.163637       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:43:42.163661       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:43:52.094940       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-2acf6ed9-7249-4fa6-9675-f4e71c4a95c6" attached to node "k8s-agentpool1-31084797-vmss000000".
... skipping 26 lines ...
I1009 02:44:07.031619       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:07.031647       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:44:07.031672       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:07.031686       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306" lun 3 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-62787193-2db7-4abf-b655-aef7e0cbc306  true false 3})]
I1009 02:44:07.031704       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:07.031734       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-62787193-2db7-4abf-b655-aef7e0cbc306  true false 3})])
I1009 02:44:07.217821       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-62787193-2db7-4abf-b655-aef7e0cbc306  true false 3})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:44:13.991319       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:44:13.991348       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-aaec181d-b96b-4280-9361-eab52a2a07d5","csi.storage.k8s.io/pvc/name":"inline-volume-tester2-jmvp4-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-5454","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5"}
I1009 02:44:14.014440       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:44:14.014471       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.014513       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:44:14.014532       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.014545       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]
I1009 02:44:14.014562       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.014600       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})])
I1009 02:44:14.074083       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:44:14.074116       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:44:14.074180       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:44:14.074222       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:44:14.074266       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:44:14.074299       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:44:14.170288       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:44:14.193788       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:44:14.270129       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:44:14.270180       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.270199       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 02:44:14.270217       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.270249       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 02:44:14.339579       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:44:14.339621       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:44:14.339677       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:44:14.339716       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:44:14.339780       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:44:14.339807       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:44:14.760356       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1009 02:44:14.782356       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.782384       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:44:14.782407       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.782420       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e" lun 4 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b19575ad-ba29-4526-b1d1-8999de9c383e  true false 4})]
I1009 02:44:14.782447       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:44:14.782472       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b19575ad-ba29-4526-b1d1-8999de9c383e  true false 4})])
I1009 02:44:14.927753       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b19575ad-ba29-4526-b1d1-8999de9c383e  true false 4})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:44:17.295922       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:44:17.295965       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:44:17.295985       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"3"}}
I1009 02:44:20.129921       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 02:44:20.129946       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47"}
I1009 02:44:20.130061       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-6302b56c-ac1d-49f0-9b3d-de0f95ca6c47 from node k8s-agentpool1-31084797-vmss000000
... skipping 87 lines ...
I1009 02:45:16.211170       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a9d24570-ad6a-4920-a8de-d80c64329714  true false 0})])
I1009 02:45:16.227313       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 02:45:16.227334       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306"}
I1009 02:45:16.227429       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306 from node k8s-agentpool1-31084797-vmss000000
I1009 02:45:16.227443       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:16.227496       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:16.367651       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a9d24570-ad6a-4920-a8de-d80c64329714  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:45:16.367873       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306" from node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306:pvc-62787193-2db7-4abf-b655-aef7e0cbc306]
I1009 02:45:16.367934       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
E1009 02:45:16.431719       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-31084797-vmss000000): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306:pvc-62787193-2db7-4abf-b655-aef7e0cbc306]) not found
I1009 02:45:16.431755       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-62787193-2db7-4abf-b655-aef7e0cbc306:pvc-62787193-2db7-4abf-b655-aef7e0cbc306])
I1009 02:45:17.040394       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:45:17.040438       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b19575ad-ba29-4526-b1d1-8999de9c383e","csi.storage.k8s.io/pvc/name":"test.csi.azure.com662gm","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-2142","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e"}
... skipping 6 lines ...
I1009 02:45:18.103546       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:18.103586       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:18.103611       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:18.103625       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]
I1009 02:45:18.103649       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:18.103684       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})])
I1009 02:45:18.184107       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:18.184171       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:18.184240       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:18.184296       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:18.184343       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:18.184375       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:18.346588       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:45:18.369049       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:18.426606       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:18.426670       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:18.426693       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 02:45:18.426710       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:18.426745       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 02:45:18.514779       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:18.514810       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:18.514860       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:18.514900       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:18.514948       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:18.514978       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:25.212417       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-31084797-vmss" and instanceID "1", refreshing the cache
... skipping 13 lines ...
I1009 02:45:31.778056       1 utils.go:102] GRPC response: {}
I1009 02:45:31.778117       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:31.778139       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:45:31.868771       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e" lun 1 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b19575ad-ba29-4526-b1d1-8999de9c383e  true false 1})]
I1009 02:45:31.868822       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:31.868844       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b19575ad-ba29-4526-b1d1-8999de9c383e  true false 1})])
I1009 02:45:32.044002       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b19575ad-ba29-4526-b1d1-8999de9c383e  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:45:33.192123       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:45:33.192151       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526","csi.storage.k8s.io/pvc/name":"test.csi.azure.com22mhx","csi.storage.k8s.io/pvc/namespace":"provisioning-6500","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526"}
I1009 02:45:33.227580       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526" to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:45:33.227616       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:33.227638       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:45:33.277903       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:45:33.277970       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:33.277992       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526" lun 2 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526  true false 2})]
I1009 02:45:33.278013       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:33.278028       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526  true false 2})])
I1009 02:45:33.437819       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7b8fe7cb-8ac0-4fdc-9da5-35c8301ce526  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:45:36.232510       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:45:36.232544       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-hn872-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-7339","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a"}
I1009 02:45:36.254728       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:45:36.254762       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:36.254801       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:36.254819       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:36.254833       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:45:36.254878       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:36.254929       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:45:36.325004       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:36.325029       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:36.325107       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:36.325158       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:36.325214       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:36.325259       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:36.330763       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:45:36.354234       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:36.432202       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:36.432244       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:36.432263       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:45:36.432281       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:36.432328       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:45:36.520670       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:36.520700       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:36.520784       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:36.520817       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:36.520887       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:36.520913       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:36.528598       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:45:36.551599       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:36.664462       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:36.664504       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:36.664525       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:45:36.664542       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:36.664575       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:45:36.741505       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:36.741534       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:36.741583       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:36.741614       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:36.741653       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:36.741676       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:37.247920       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 8 lines ...
I1009 02:45:37.326667       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:45:37.331837       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:45:37.331858       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-hn872-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-7339","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a"}
I1009 02:45:37.356047       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:45:37.356075       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:37.356114       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:37.389955       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:37.389986       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:37.390035       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:37.390071       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:37.390128       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:37.390186       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:45:37.390196       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:37.390211       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
... skipping 2 lines ...
I1009 02:45:37.419878       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:45:37.419922       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:37.447522       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:45:37.447554       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:37.447589       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:45:37.447714       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:37.508141       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:37.508168       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:37.508239       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:37.508287       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:37.508348       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:37.508378       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:37.508449       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:37.508487       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:37.590167       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:45:37.590208       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:37.590247       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:45:37.651853       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:37.651916       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:37.651998       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:37.652047       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:37.652088       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:37.652127       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:38.397516       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:45:38.430174       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:38.484847       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:38.484903       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:38.484941       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:45:38.484959       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:38.485006       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:45:38.546925       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:38.546952       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:38.547051       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:38.547092       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:38.547134       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:38.547160       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:42.554998       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:45:42.581054       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:42.713958       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:42.713999       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:42.714018       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:45:42.714033       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:42.714083       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:45:42.819912       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:42.819947       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:42.820010       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:42.820044       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:42.820085       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:42.820113       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:45.513564       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:45:45.546900       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:45.619662       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:45.619704       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:45.619725       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:45:45.619820       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:45.619866       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:45:45.688442       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:45.688494       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:45.688598       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:45.688644       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:45.688715       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:45.688746       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:50.559695       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 11 lines ...
I1009 02:45:50.910257       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:45:50.983689       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:45:50.983803       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:50.983842       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:45:50.983865       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:45:50.983897       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:45:51.058485       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:51.058515       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:45:51.058588       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:45:51.058666       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:45:51.058712       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:45:51.058767       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:45:59.327833       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 8 lines ...
I1009 02:46:01.722137       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:46:01.790576       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:46:01.790623       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:01.790643       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:46:01.790659       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:01.790710       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:46:01.850482       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:01.850539       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:46:01.850595       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:46:01.850650       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:46:01.850698       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:46:01.850729       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:05.810661       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-a9d24570-ad6a-4920-a8de-d80c64329714:pvc-a9d24570-ad6a-4920-a8de-d80c64329714]) returned with <nil>
... skipping 2 lines ...
I1009 02:46:05.810804       1 utils.go:102] GRPC response: {}
I1009 02:46:05.810936       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:05.810973       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:46:05.858783       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-253a80cc-1fe0-4e8b-a303-f90c10475177" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-253a80cc-1fe0-4e8b-a303-f90c10475177:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-253a80cc-1fe0-4e8b-a303-f90c10475177  true false 0})]
I1009 02:46:05.858824       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:05.858844       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-253a80cc-1fe0-4e8b-a303-f90c10475177:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-253a80cc-1fe0-4e8b-a303-f90c10475177  true false 0})])
I1009 02:46:06.003876       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-253a80cc-1fe0-4e8b-a303-f90c10475177:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-253a80cc-1fe0-4e8b-a303-f90c10475177  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:46:07.068044       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:46:07.068069       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qwk8d-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-526","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d"}
I1009 02:46:07.091894       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:46:07.091926       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:07.091984       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:46:07.092003       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:07.092016       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:46:07.092069       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:07.092102       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:46:07.156273       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:07.156304       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:46:07.156353       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:46:07.156382       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:46:07.156414       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:46:07.156443       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:07.250401       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e" attached to node "k8s-agentpool1-31084797-vmss000000".
... skipping 19 lines ...
I1009 02:46:33.886396       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:33.886436       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:46:33.886466       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:33.886479       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:46:33.886507       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:33.886542       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:46:33.963856       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:33.963890       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:46:33.963941       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:46:33.964024       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:46:33.964077       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:46:33.964113       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:39.166745       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:46:39.191382       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:46:39.256815       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:46:39.256862       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:39.256892       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:46:39.256909       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:46:39.256941       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:46:39.344588       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:39.344615       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:46:39.344666       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:46:39.344696       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:46:39.344727       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:46:39.344750       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:46:40.903000       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 28 lines ...
I1009 02:47:26.218428       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:26.218470       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:47:26.218494       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:26.218507       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]
I1009 02:47:26.218553       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:26.218602       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})])
I1009 02:47:26.281343       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:26.281373       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:47:26.281423       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:47:26.281455       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:47:26.281503       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:47:26.281527       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:26.522243       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:47:26.546562       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:47:26.597655       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:47:26.597713       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:26.597732       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 02:47:26.597749       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:26.597812       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 02:47:26.659623       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:26.659651       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:47:26.659703       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:47:26.659744       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:47:26.659808       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:47:26.659832       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:31.164092       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 16 lines ...
I1009 02:47:38.004250       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:47:38.080450       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:47:38.080513       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:38.080534       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:47:38.080552       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:38.080582       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:47:38.143400       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:38.143428       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:47:38.143512       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:47:38.143557       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:47:38.143605       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:47:38.143635       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:43.358736       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:47:43.397722       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:47:43.464378       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:47:43.464430       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:43.464447       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:47:43.464462       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:43.464506       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:47:43.544430       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:43.544475       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:47:43.544526       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:47:43.544588       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:47:43.544629       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:47:43.544671       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:47:51.515387       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b19575ad-ba29-4526-b1d1-8999de9c383e:pvc-b19575ad-ba29-4526-b1d1-8999de9c383e]) returned with <nil>
... skipping 2 lines ...
I1009 02:47:51.515540       1 utils.go:102] GRPC response: {}
I1009 02:47:51.515668       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:51.515691       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:47:51.578182       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f8058fe6-61ba-4b61-a829-20311e63f16d  true false 0})]
I1009 02:47:51.578225       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:47:51.578264       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f8058fe6-61ba-4b61-a829-20311e63f16d  true false 0})])
I1009 02:47:51.758323       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f8058fe6-61ba-4b61-a829-20311e63f16d  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:48:01.864488       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:48:01.864529       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:48:01.864564       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:48:01.871035       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:48:01.871057       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f8058fe6-61ba-4b61-a829-20311e63f16d","csi.storage.k8s.io/pvc/name":"test.csi.azure.comb8lxw","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-4743","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d"}
I1009 02:48:01.897285       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-f8058fe6-61ba-4b61-a829-20311e63f16d" to node "k8s-agentpool1-31084797-vmss000000".
... skipping 8 lines ...
I1009 02:48:03.601425       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:48:03.601468       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:48:03.601501       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:48:03.601513       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2" lun 1 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2  true false 1})]
I1009 02:48:03.601534       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:48:03.601550       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2  true false 1})])
I1009 02:48:03.778313       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:48:10.627197       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:48:10.627229       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad","csi.storage.k8s.io/pvc/name":"test.csi.azure.com8d28f","csi.storage.k8s.io/pvc/namespace":"provisioning-5586","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad"}
I1009 02:48:10.651006       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad" to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:48:10.651054       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:48:10.651075       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:48:10.716641       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:48:10.716685       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:48:10.716760       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad" lun 2 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad  true false 2})]
I1009 02:48:10.716793       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:48:10.716809       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad  true false 2})])
I1009 02:48:10.873482       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad  true false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:48:20.950791       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:48:20.950834       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-0a548415-ac5c-4c06-b2ad-05157eb2f6ad" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:48:20.950854       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"2"}}
I1009 02:48:23.925679       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:48:23.925721       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-09dbd332-ca84-4c6d-bc00-d4ef0b67a0d2" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:48:23.925744       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 42 lines ...
I1009 02:49:46.181023       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:46.181098       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:49:46.181166       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:46.181178       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:49:46.181193       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:46.181242       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:49:46.246276       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:49:46.246304       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:49:46.246375       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:49:46.246432       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:49:46.246488       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:49:46.246510       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:49:51.557234       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:49:51.581804       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:49:51.664202       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:49:51.664242       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:51.664262       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:49:51.664280       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:51.664312       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:49:51.728394       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:49:51.728440       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:49:51.728489       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:49:51.728519       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:49:51.728560       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:49:51.728594       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:49:58.406898       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 2 lines ...
I1009 02:49:58.429690       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:58.429736       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:49:58.429772       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:58.429784       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5b823d6-896c-4843-ba05-2359f195edfb  true false 0})]
I1009 02:49:58.429819       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:49:58.429834       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5b823d6-896c-4843-ba05-2359f195edfb  true false 0})])
I1009 02:49:58.590894       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5b823d6-896c-4843-ba05-2359f195edfb  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:50:13.695970       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:50:13.696008       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:50:13.696029       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:50:25.920009       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-31084797-vmss" and instanceID "1", refreshing the cache
I1009 02:50:32.412069       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 02:50:32.412097       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c5b823d6-896c-4843-ba05-2359f195edfb"}
... skipping 14 lines ...
I1009 02:51:12.640689       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:51:13.643023       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:51:13.643079       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:13.643101       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2  true false 0})]
I1009 02:51:13.643123       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:13.643138       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2  true false 0})])
I1009 02:51:14.883417       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:51:24.481693       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:51:24.481693       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:51:24.481731       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb","csi.storage.k8s.io/pvc/name":"pvc-5zqqk","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb"}
I1009 02:51:24.481690       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:51:24.481801       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d","csi.storage.k8s.io/pvc/name":"inline-volume-tester-qwk8d-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-526","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d"}
I1009 02:51:24.481917       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-hn872-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-7339","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a"}
... skipping 12 lines ...
I1009 02:51:24.803250       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:51:24.803288       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:24.803335       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:51:24.814097       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:51:24.814134       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:24.814175       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:51:24.876046       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:51:24.876080       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:51:24.876133       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:51:24.876171       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:51:24.876225       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:51:24.876253       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:51:24.876310       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:24.876333       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:51:24.978877       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 34 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 34})]
I1009 02:51:24.978940       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:24.979010       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 34})])
I1009 02:51:25.052803       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:51:25.052834       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:51:25.052901       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 34})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:51:25.052960       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 33}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 34})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:51:25.053021       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:51:25.053051       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:51:25.053120       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:25.053155       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
E1009 02:51:25.104260       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d) in current disk list(len: 32) nor in diskMap(map[])
E1009 02:51:25.104375       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d) in current disk list(len: 32) nor in diskMap(map[])
I1009 02:51:25.104441       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 02:51:25.104511       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 32) nor in diskMap(map[])
E1009 02:51:25.104545       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5) in current disk list(len: 32) nor in diskMap(map[])
I1009 02:51:29.977188       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:51:29.977246       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:51:29.977265       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:51:29.993041       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:51:29.993069       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2","csi.storage.k8s.io/pvc/name":"test.csi.azure.com9bwpq","csi.storage.k8s.io/pvc/namespace":"fsgroupchangepolicy-5961","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2"}
I1009 02:51:30.025698       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-7b55aed0-9319-43da-a03a-402e2ea8d9d2" to node "k8s-agentpool1-31084797-vmss000000".
... skipping 7 lines ...
I1009 02:51:42.318805       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:42.348562       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:51:42.348609       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:42.348626       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]
I1009 02:51:42.348643       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:42.348676       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})])
I1009 02:51:42.413617       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:51:42.413659       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:51:42.413723       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:51:42.413782       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:51:42.413818       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:51:42.413846       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:51:42.665878       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:51:42.707644       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:51:42.777785       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:51:42.777827       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:42.777866       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 02:51:42.777885       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:51:42.777916       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 02:51:42.840767       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:51:42.840797       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:51:42.840847       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:51:42.840880       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:51:42.840917       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:51:42.840943       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:52:13.071868       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 16 lines ...
I1009 02:53:01.572745       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:53:01.637573       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:53:01.637620       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:53:01.637642       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f  true false 0})]
I1009 02:53:01.637659       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:53:01.637671       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f  true false 0})])
I1009 02:53:01.924054       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:53:12.002639       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:53:12.002676       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:53:12.002711       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:53:26.530913       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-31084797-vmss" and instanceID "0", refreshing the cache
I1009 02:54:02.260411       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:54:02.260450       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-hn872-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-7339","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a"}
I1009 02:54:02.284313       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:54:02.284354       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:02.284412       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:54:02.284440       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:02.284463       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]
I1009 02:54:02.284499       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:02.284536       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})])
I1009 02:54:02.364618       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:54:02.364664       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:54:02.364717       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:54:02.364760       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:54:02.364804       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:54:02.364836       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:54:03.842134       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
... skipping 11 lines ...
I1009 02:54:07.763159       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:54:07.830869       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:54:07.830914       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:07.830938       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:54:07.830955       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:07.830986       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:54:07.901593       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:54:07.901643       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:54:07.901708       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:54:07.901758       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:54:07.901792       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:54:07.901825       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:54:24.118852       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f:pvc-aba1e8dd-d2bd-4b06-8c90-7dabfaad422f]) returned with <nil>
... skipping 7 lines ...
I1009 02:54:37.557683       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:37.557740       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:54:37.557755       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:37.557767       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96163c7a-f6bb-4f28-ba3e-02734349125d  true false 0})]
I1009 02:54:37.557784       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:54:37.557797       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96163c7a-f6bb-4f28-ba3e-02734349125d  true false 0})])
I1009 02:54:37.800865       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-96163c7a-f6bb-4f28-ba3e-02734349125d  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:54:47.902742       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:54:47.902785       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:54:47.902805       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:55:04.245279       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 02:55:04.245306       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d"}
I1009 02:55:04.245410       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-96163c7a-f6bb-4f28-ba3e-02734349125d from node k8s-agentpool1-31084797-vmss000000
... skipping 14 lines ...
I1009 02:55:51.684016       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:55:51.754551       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:55:51.754611       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:55:51.754630       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360  true false 0})]
I1009 02:55:51.754649       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:55:51.754662       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360  true false 0})])
I1009 02:55:51.949954       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:56:02.027640       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:56:02.027685       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:56:02.027706       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:56:24.752722       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 02:56:24.752753       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360"}
I1009 02:56:24.752864       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360 from node k8s-agentpool1-31084797-vmss000000
... skipping 14 lines ...
I1009 02:56:42.462908       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:42.462947       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:56:42.462965       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:42.462978       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]
I1009 02:56:42.462994       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:42.463027       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})])
I1009 02:56:42.528240       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:56:42.528277       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:56:42.528330       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:56:42.528373       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-aaec181d-b96b-4280-9361-eab52a2a07d5  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:56:42.528418       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:56:42.528443       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-aaec181d-b96b-4280-9361-eab52a2a07d5" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:56:42.847496       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 3 lines ...
I1009 02:56:42.872813       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 02:56:42.977337       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:56:42.977387       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:42.977410       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 02:56:42.977428       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:42.977459       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 02:56:43.042022       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:56:43.042058       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:56:43.042110       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:56:43.042148       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:56:43.042187       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:56:43.042218       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:56:45.138869       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360:pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360]) returned with <nil>
... skipping 7 lines ...
I1009 02:56:45.149580       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360 from node k8s-agentpool1-31084797-vmss000000
I1009 02:56:45.149599       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:45.200246       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab  true false 0})]
I1009 02:56:45.200289       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:45.200308       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab  true false 0})])
I1009 02:56:45.200367       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:56:45.366782       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:56:45.366960       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360" from node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360:pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360]
I1009 02:56:45.367007       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
E1009 02:56:45.555715       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-31084797-vmss000000): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360:pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360]) not found
I1009 02:56:45.555777       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360:pvc-23d47ea6-1cc6-49bf-a664-856c23ce8360])
I1009 02:57:00.460691       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:57:00.460733       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e94c4085-c53e-4b9b-87c6-6703f8824aab" to node "k8s-agentpool1-31084797-vmss000000" successfully
... skipping 34 lines ...
I1009 02:58:39.557911       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:58:39.557934       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:58:39.557952       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:58:39.557964       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9538ceb7-32b1-4297-b2df-dfc68992b508  true false 0})]
I1009 02:58:39.557984       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:58:39.557999       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9538ceb7-32b1-4297-b2df-dfc68992b508  true false 0})])
I1009 02:58:39.928629       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9538ceb7-32b1-4297-b2df-dfc68992b508  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:58:50.015714       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 02:58:50.015755       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 02:58:50.015801       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 02:58:50.022658       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:58:50.022680       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9538ceb7-32b1-4297-b2df-dfc68992b508","csi.storage.k8s.io/pvc/name":"test.csi.azure.comcv8k8","csi.storage.k8s.io/pvc/namespace":"provisioning-4773","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508"}
I1009 02:58:50.049955       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508" to node "k8s-agentpool1-31084797-vmss000000".
... skipping 8 lines ...
I1009 02:58:54.185002       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:58:54.185030       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:58:54.185048       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:58:54.185061       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499" lun 1 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499  true false 1})]
I1009 02:58:54.185081       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:58:54.185098       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499  true false 1})])
I1009 02:58:54.362167       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:59:02.379151       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 02:59:02.379179       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a","csi.storage.k8s.io/pvc/name":"inline-volume-tester-hn872-my-volume-0","csi.storage.k8s.io/pvc/namespace":"ephemeral-7339","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a"}
I1009 02:59:02.424686       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001".
I1009 02:59:02.424730       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:02.424778       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:59:02.424796       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:02.424810       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a" lun 1 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 1})]
I1009 02:59:02.424834       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:02.424869       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 1})])
I1009 02:59:02.621625       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c25c768c-a563-40ab-acc2-c18c0d1cbf7a  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 02:59:05.736553       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 02:59:05.736582       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508"}
I1009 02:59:05.736691       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508 from node k8s-agentpool1-31084797-vmss000000
I1009 02:59:05.736715       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:05.736738       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:59:05.819223       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
... skipping 6 lines ...
I1009 02:59:07.942275       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:07.942318       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to node "k8s-agentpool1-31084797-vmss000001"
I1009 02:59:07.942336       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:07.942349       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 32 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]
I1009 02:59:07.942373       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:07.942406       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})])
I1009 02:59:08.005906       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:59:08.005935       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 02:59:08.006002       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 02:59:08.006053       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 02:59:08.006125       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 02:59:08.006177       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 02:59:26.080650       1 azure_controller_vmss.go:225] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-9538ceb7-32b1-4297-b2df-dfc68992b508:pvc-9538ceb7-32b1-4297-b2df-dfc68992b508]) returned with <nil>
... skipping 40 lines ...
I1009 02:59:50.740327       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 02:59:50.810217       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226" to node "k8s-agentpool1-31084797-vmss000000"
I1009 02:59:50.810254       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:50.810289       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226  true false 0})]
I1009 02:59:50.810321       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 02:59:50.810351       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226  true false 0})])
I1009 02:59:50.966405       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:00:01.068872       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 03:00:01.068947       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b4abb01b-1be0-42a6-b8c3-4870b5d91226" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 03:00:01.068975       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 03:00:06.173017       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 03:00:06.173044       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:06.173150       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499 from node k8s-agentpool1-31084797-vmss000000
... skipping 40 lines ...
I1009 03:00:41.875533       1 utils.go:102] GRPC response: {}
I1009 03:00:41.875571       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:00:41.875603       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 03:00:41.942894       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499  true false 0})]
I1009 03:00:41.942942       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:00:41.942981       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499  true false 0})])
I1009 03:00:42.098747       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:00:52.199968       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 03:00:52.200009       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 03:00:52.200028       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 03:00:52.208437       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 03:00:52.208459       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499","csi.storage.k8s.io/pvc/name":"test.csi.azure.comvnf9x","csi.storage.k8s.io/pvc/namespace":"volume-expand-1286","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499"}
I1009 03:00:52.242510       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499" to node "k8s-agentpool1-31084797-vmss000000".
... skipping 38 lines ...
I1009 03:01:42.143451       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" lun 1 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]
I1009 03:01:42.143488       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:42.143542       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})])
I1009 03:01:42.143968       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499) succeeded
I1009 03:01:42.143997       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-25e94c33-4abf-4a69-bc8e-51a12d40b499 from node k8s-agentpool1-31084797-vmss000000 successfully
I1009 03:01:42.144012       1 utils.go:102] GRPC response: {}
I1009 03:01:42.232845       1 azure_armclient.go:341] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1?api-version=2020-12-01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 03:01:42.232878       1 azure_armclient.go:594] Received error in put.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-31084797-vmss/virtualMachines/1, error: %!s(<nil>)
E1009 03:01:42.232929       1 azure_controller_vmss.go:125] azureDisk - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})]) on rg(kubetest-enl5pco2) vm(k8s-agentpool1-31084797-vmss000001) failed, err: &{false 409 0001-01-01 00:00:00 +0000 UTC {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}
I1009 03:01:42.232964       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d  true false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 32})], &{%!s(bool=false) %!s(int=409) 0001-01-01 00:00:00 +0000 UTC {
  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}}) returned with %!v(MISSING)
E1009 03:01:42.233048       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
E1009 03:01:42.233081       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" to instance "k8s-agentpool1-31084797-vmss000001" failed with Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {

  "error": {

    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 32.",
    "target": "dataDisks"
  }
}
I1009 03:01:42.233156       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:42.233184       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 03:01:42.238759       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 03:01:42.238785       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d"}
I1009 03:01:42.238927       1 controllerserver.go:524] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d from node k8s-agentpool1-31084797-vmss000001
I1009 03:01:42.238952       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
E1009 03:01:42.319906       1 controllerserver.go:481] Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 31) nor in diskMap(map[])
E1009 03:01:42.319954       1 utils.go:100] GRPC error: Attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to instance "k8s-agentpool1-31084797-vmss000001" failed with could not find disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb) in current disk list(len: 31) nor in diskMap(map[])
I1009 03:01:42.319980       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:42.320006       1 azure_controller_common.go:346] Trying to detach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d" from node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d]
E1009 03:01:42.320062       1 azure_controller_vmss.go:183] detach azure disk on node(k8s-agentpool1-31084797-vmss000001): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d]) not found
I1009 03:01:42.320080       1 azure_controller_vmss.go:213] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d:pvc-d3a8cfac-6c52-421e-8fb8-737d4917ad7d])
I1009 03:01:43.048718       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 03:01:43.048753       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb","csi.storage.k8s.io/pvc/name":"pvc-5zqqk","csi.storage.k8s.io/pvc/namespace":"volumelimits-9466","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb"}
... skipping 6 lines ...
I1009 03:01:47.514841       1 utils.go:102] GRPC response: {}
I1009 03:01:47.514902       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:47.514924       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 03:01:47.575244       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" lun 1 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 1})]
I1009 03:01:47.575294       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:47.575334       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 1})])
I1009 03:01:47.761471       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:01:58.773115       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
I1009 03:01:58.773169       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000000","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3","csi.storage.k8s.io/pvc/name":"test.csi.azure.comlbz57","csi.storage.k8s.io/pvc/namespace":"provisioning-9114","requestedsizegib":"5","storage.kubernetes.io/csiProvisionerIdentity":"1633747283416-8081-test.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3"}
I1009 03:01:58.796571       1 controllerserver.go:440] GetDiskLun returned: <nil>. Initiating attaching volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3" to node "k8s-agentpool1-31084797-vmss000000".
I1009 03:01:58.796603       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:58.796627       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3" to node "k8s-agentpool1-31084797-vmss000000"
I1009 03:01:58.796646       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:58.796658       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3  true false 0})]
I1009 03:01:58.796674       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:01:58.796688       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3  true false 0})])
I1009 03:01:58.952945       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:02:17.956468       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 03:02:17.956511       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-d827dda7-6c90-45ce-9769-5c52119ca6cb" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 03:02:17.956532       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
I1009 03:02:19.066173       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 03:02:19.066226       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-af743ad7-58f4-4d3c-966f-654c82b8ece3" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 03:02:19.066248       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
... skipping 22 lines ...
I1009 03:03:27.079691       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5" to node "k8s-agentpool1-31084797-vmss000000"
I1009 03:03:27.079733       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:03:27.079781       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5" lun 0 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5  true false 0})]
I1009 03:03:27.079806       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:03:27.079823       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5  true false 0})])
I1009 03:03:27.079859       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8" to node "k8s-agentpool1-31084797-vmss000000"
I1009 03:03:27.259603       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:03:27.259810       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:03:27.259837       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000000, refreshing the cache
I1009 03:03:27.343745       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8" lun 1 to node "k8s-agentpool1-31084797-vmss000000", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c3497036-aca4-4f6f-b472-cef20212dfb8  true false 1})]
I1009 03:03:27.343823       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000000" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:03:27.343848       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c3497036-aca4-4f6f-b472-cef20212dfb8  true false 1})])
I1009 03:03:27.707211       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c3497036-aca4-4f6f-b472-cef20212dfb8  true false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:03:42.400958       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 03:03:42.401022       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-b503d73d-686d-4bb6-9190-894d7a2a9dd5" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 03:03:42.401054       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 03:03:42.824002       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8" attached to node "k8s-agentpool1-31084797-vmss000000".
I1009 03:03:42.824041       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-c3497036-aca4-4f6f-b472-cef20212dfb8" to node "k8s-agentpool1-31084797-vmss000000" successfully
I1009 03:03:42.824059       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"1"}}
... skipping 537 lines ...
I1009 03:06:55.240569       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:06:55.240598       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a" to node "k8s-agentpool1-31084797-vmss000001"
I1009 03:06:55.240620       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:06:55.240633       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a" lun 0 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e90ee62f-f04e-4958-a67e-099082dfd89a  true false 0})]
I1009 03:06:55.240658       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:06:55.240674       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e90ee62f-f04e-4958-a67e-099082dfd89a  true false 0})])
I1009 03:06:55.450189       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e90ee62f-f04e-4958-a67e-099082dfd89a  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:07:05.560402       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 03:07:05.560471       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 03:07:05.560523       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0"}}
I1009 03:07:28.704647       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-31084797-vmss" and instanceID "1", refreshing the cache
I1009 03:07:49.305169       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume
I1009 03:07:49.305196       1 utils.go:96] GRPC request: {"node_id":"k8s-agentpool1-31084797-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-e90ee62f-f04e-4958-a67e-099082dfd89a"}
... skipping 27 lines ...
I1009 03:08:27.106598       1 azure_vmss.go:198] Couldn't find VMSS VM with nodeName k8s-agentpool1-31084797-vmss000001, refreshing the cache
I1009 03:08:27.178522       1 controllerserver.go:461] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea" to node "k8s-agentpool1-31084797-vmss000001"
I1009 03:08:27.178567       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:08:27.178588       1 azure_controller_common.go:242] Trying to attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea" lun 0 to node "k8s-agentpool1-31084797-vmss000001", diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea  true false 0})]
I1009 03:08:27.178607       1 azure_vmss_cache.go:320] Assuming node "k8s-agentpool1-31084797-vmss000001" is managed by VMSS since DisableAvailabilitySetNodes is set to true
I1009 03:08:27.178620       1 azure_controller_vmss.go:122] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea  true false 0})])
I1009 03:08:27.348457       1 azure_controller_vmss.go:134] azureDisk - update(kubetest-enl5pco2): vm(k8s-agentpool1-31084797-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-enl5pco2/providers/microsoft.compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea  true false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING)
I1009 03:08:28.875957       1 azure_vmss.go:319] Couldn't find VMSS VM with scaleSetName "k8s-agentpool1-31084797-vmss" and instanceID "1", refreshing the cache
I1009 03:08:37.448615       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea" attached to node "k8s-agentpool1-31084797-vmss000001".
I1009 03:08:37.448658       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-enl5pco2/providers/Microsoft.Compute/disks/pvc-90b4629d-548f-40d6-b568-f2d4a013d4ea" to node "k8s-agentpool1-31084797-vmss000001" successfully
I1009 03:08:37.448680       1 utils.go:102] GRPC response: {"publish_context":{