kubernetes-sigs/azuredisk-csi-driver PR #1685 pull-azuredisk-csi-driver-e2e-multi-az #1612352130994147328
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| PR | andyzhangx: [WIP] test |
| Result | FAILURE |
| Tests | 1 failed / 13 succeeded |
| Started | |
| Elapsed | 1h26m |
| Revision | 0f6f26196850f19d8bc2d41d0c07658694c5896f |
| Refs |
1685 |
| job-version | v1.27.0-alpha.0.834+cf7a3c5bbbebf2 |
| kubetest-version | v20230105-c1beb6cf98 |
| revision | v1.27.0-alpha.0.834+cf7a3c5bbbebf2 |
error during make e2e-test: exit status 2
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest Check APIReachability
kubetest Deferred TearDown
kubetest DumpClusterLogs
kubetest GetDeployer
kubetest IsUp
kubetest Prepare
kubetest TearDown
kubetest TearDown Previous
kubetest Timeout
kubetest Up
kubetest kubectl version
kubetest list nodes
kubetest test setup
... skipping 107 lines ... 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 11345 100 11345 0 0 120k 0 --:--:-- --:--:-- --:--:-- 120k Downloading https://get.helm.sh/helm-v3.10.3-linux-amd64.tar.gz Verifying checksum... Done. Preparing to install helm into /usr/local/bin helm installed into /usr/local/bin/helm docker pull k8sprow.azurecr.io/azuredisk-csi:v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 || make container-all push-manifest Error response from daemon: manifest for k8sprow.azurecr.io/azuredisk-csi:v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 not found: manifest unknown: manifest tagged by "v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7" is not found make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver' CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=22e27a8ea5922e24a5393b23757f258026dbfee7 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2023-01-09T07:47:30Z -extldflags "-static"" -mod vendor -o _output/amd64/azurediskplugin.exe ./pkg/azurediskplugin docker buildx rm container-builder || true ERROR: no builder "container-builder" found docker buildx create --use --name=container-builder container-builder # enable qemu for arm64 build # https://github.com/docker/buildx/issues/464#issuecomment-741507760 docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64 Unable to find image 'tonistiigi/binfmt:latest' locally ... skipping 1762 lines ... type: string type: object oneOf: - required: ["persistentVolumeClaimName"] - required: ["volumeSnapshotContentName"] volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 60 lines ... type: string volumeSnapshotContentName: description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable. type: string type: object volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 254 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 108 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 865 lines ... image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.6.0" args: - "-csi-address=$(ADDRESS)" - "-v=2" - "-leader-election" - "--leader-election-namespace=kube-system" - '-handle-volume-inuse-error=false' - '-feature-gates=RecoverVolumeExpansionFailure=true' - "-timeout=240s" env: - name: ADDRESS value: /csi/csi.sock volumeMounts: ... skipping 216 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:04:28.306[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:04:28.308[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:04:28.367[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:04:28.367[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:04:28.431[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:04:28.432[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:04:28.496[0m Jan 9 08:04:28.497: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-97sff" in namespace "azuredisk-8081" to be "Succeeded or Failed" Jan 9 08:04:28.554: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 56.859771ms Jan 9 08:04:30.622: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 2.125284551s Jan 9 08:04:32.614: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 4.117268799s Jan 9 08:04:34.613: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 6.116098889s Jan 9 08:04:36.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 8.114681229s Jan 9 08:04:38.612: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115347061s ... skipping 7 lines ... Jan 9 08:04:54.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 26.114431991s Jan 9 08:04:56.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 28.11404785s Jan 9 08:04:58.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 30.114250672s Jan 9 08:05:00.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 32.113824573s Jan 9 08:05:02.613: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Succeeded", Reason="", readiness=false. Elapsed: 34.116359747s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:05:02.613[0m Jan 9 08:05:02.615: INFO: Pod "azuredisk-volume-tester-97sff" satisfied condition "Succeeded or Failed" Jan 9 08:05:02.615: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-97sff" Jan 9 08:05:02.722: INFO: Pod azuredisk-volume-tester-97sff has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-97sff in namespace azuredisk-8081 [38;5;243m01/09/23 08:05:02.722[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:05:02.844[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:05:02.901[0m ... skipping 57 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:04:28.306[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:04:28.308[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:04:28.367[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:04:28.367[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:04:28.431[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:04:28.432[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:04:28.496[0m Jan 9 08:04:28.497: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-97sff" in namespace "azuredisk-8081" to be "Succeeded or Failed" Jan 9 08:04:28.554: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 56.859771ms Jan 9 08:04:30.622: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 2.125284551s Jan 9 08:04:32.614: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 4.117268799s Jan 9 08:04:34.613: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 6.116098889s Jan 9 08:04:36.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 8.114681229s Jan 9 08:04:38.612: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115347061s ... skipping 7 lines ... Jan 9 08:04:54.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 26.114431991s Jan 9 08:04:56.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 28.11404785s Jan 9 08:04:58.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 30.114250672s Jan 9 08:05:00.611: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Pending", Reason="", readiness=false. Elapsed: 32.113824573s Jan 9 08:05:02.613: INFO: Pod "azuredisk-volume-tester-97sff": Phase="Succeeded", Reason="", readiness=false. Elapsed: 34.116359747s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:05:02.613[0m Jan 9 08:05:02.615: INFO: Pod "azuredisk-volume-tester-97sff" satisfied condition "Succeeded or Failed" Jan 9 08:05:02.615: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-97sff" Jan 9 08:05:02.722: INFO: Pod azuredisk-volume-tester-97sff has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-97sff in namespace azuredisk-8081 [38;5;243m01/09/23 08:05:02.722[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:05:02.844[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:05:02.901[0m ... skipping 39 lines ... Jan 9 08:05:46.820: INFO: PersistentVolumeClaim pvc-jw64p found but phase is Pending instead of Bound. Jan 9 08:05:48.879: INFO: PersistentVolumeClaim pvc-jw64p found and phase=Bound (4.170740883s) [1mSTEP:[0m checking the PVC [38;5;243m01/09/23 08:05:48.879[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:05:48.937[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:05:48.993[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:05:48.993[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m01/09/23 08:05:49.052[0m Jan 9 08:05:49.052: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fcx4r" in namespace "azuredisk-2540" to be "Succeeded or Failed" Jan 9 08:05:49.108: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 56.205944ms Jan 9 08:05:51.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113505631s Jan 9 08:05:53.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 4.113586744s Jan 9 08:05:55.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 6.11250131s Jan 9 08:05:57.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 8.112951327s Jan 9 08:05:59.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114241263s Jan 9 08:06:01.167: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 12.114782597s Jan 9 08:06:03.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 14.113606219s Jan 9 08:06:05.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 16.113175396s Jan 9 08:06:07.167: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 18.11450635s Jan 9 08:06:09.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Succeeded", Reason="", readiness=false. Elapsed: 20.112426564s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:06:09.165[0m Jan 9 08:06:09.165: INFO: Pod "azuredisk-volume-tester-fcx4r" satisfied condition "Succeeded or Failed" Jan 9 08:06:09.165: INFO: deleting Pod "azuredisk-2540"/"azuredisk-volume-tester-fcx4r" Jan 9 08:06:09.259: INFO: Pod azuredisk-volume-tester-fcx4r has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-fcx4r in namespace azuredisk-2540 [38;5;243m01/09/23 08:06:09.259[0m Jan 9 08:06:09.323: INFO: deleting PVC "azuredisk-2540"/"pvc-jw64p" Jan 9 08:06:09.323: INFO: Deleting PersistentVolumeClaim "pvc-jw64p" ... skipping 38 lines ... Jan 9 08:05:46.820: INFO: PersistentVolumeClaim pvc-jw64p found but phase is Pending instead of Bound. Jan 9 08:05:48.879: INFO: PersistentVolumeClaim pvc-jw64p found and phase=Bound (4.170740883s) [1mSTEP:[0m checking the PVC [38;5;243m01/09/23 08:05:48.879[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:05:48.937[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:05:48.993[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:05:48.993[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m01/09/23 08:05:49.052[0m Jan 9 08:05:49.052: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fcx4r" in namespace "azuredisk-2540" to be "Succeeded or Failed" Jan 9 08:05:49.108: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 56.205944ms Jan 9 08:05:51.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113505631s Jan 9 08:05:53.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 4.113586744s Jan 9 08:05:55.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 6.11250131s Jan 9 08:05:57.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 8.112951327s Jan 9 08:05:59.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114241263s Jan 9 08:06:01.167: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 12.114782597s Jan 9 08:06:03.166: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 14.113606219s Jan 9 08:06:05.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 16.113175396s Jan 9 08:06:07.167: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Pending", Reason="", readiness=false. Elapsed: 18.11450635s Jan 9 08:06:09.165: INFO: Pod "azuredisk-volume-tester-fcx4r": Phase="Succeeded", Reason="", readiness=false. Elapsed: 20.112426564s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:06:09.165[0m Jan 9 08:06:09.165: INFO: Pod "azuredisk-volume-tester-fcx4r" satisfied condition "Succeeded or Failed" Jan 9 08:06:09.165: INFO: deleting Pod "azuredisk-2540"/"azuredisk-volume-tester-fcx4r" Jan 9 08:06:09.259: INFO: Pod azuredisk-volume-tester-fcx4r has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-fcx4r in namespace azuredisk-2540 [38;5;243m01/09/23 08:06:09.259[0m Jan 9 08:06:09.323: INFO: deleting PVC "azuredisk-2540"/"pvc-jw64p" Jan 9 08:06:09.323: INFO: Deleting PersistentVolumeClaim "pvc-jw64p" ... skipping 30 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:06:50.956[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:06:50.956[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:06:51.014[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:06:51.014[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:06:51.074[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:06:51.074[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:06:51.132[0m Jan 9 08:06:51.133: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7zjng" in namespace "azuredisk-4728" to be "Succeeded or Failed" Jan 9 08:06:51.190: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 57.242647ms Jan 9 08:06:53.248: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114984426s Jan 9 08:06:55.248: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 4.115286284s Jan 9 08:06:57.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 6.116319772s Jan 9 08:06:59.247: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 8.11491878s Jan 9 08:07:01.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 10.116531676s ... skipping 18 lines ... Jan 9 08:07:39.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 48.116851552s Jan 9 08:07:41.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 50.116056978s Jan 9 08:07:43.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 52.116176641s Jan 9 08:07:45.247: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Running", Reason="", readiness=true. Elapsed: 54.114844048s Jan 9 08:07:47.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Succeeded", Reason="", readiness=false. Elapsed: 56.116223892s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:07:47.249[0m Jan 9 08:07:47.249: INFO: Pod "azuredisk-volume-tester-7zjng" satisfied condition "Succeeded or Failed" Jan 9 08:07:47.249: INFO: deleting Pod "azuredisk-4728"/"azuredisk-volume-tester-7zjng" Jan 9 08:07:47.343: INFO: Pod azuredisk-volume-tester-7zjng has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7zjng in namespace azuredisk-4728 [38;5;243m01/09/23 08:07:47.343[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:07:47.478[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:07:47.535[0m ... skipping 33 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:06:50.956[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:06:50.956[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:06:51.014[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:06:51.014[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:06:51.074[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:06:51.074[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:06:51.132[0m Jan 9 08:06:51.133: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7zjng" in namespace "azuredisk-4728" to be "Succeeded or Failed" Jan 9 08:06:51.190: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 57.242647ms Jan 9 08:06:53.248: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114984426s Jan 9 08:06:55.248: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 4.115286284s Jan 9 08:06:57.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 6.116319772s Jan 9 08:06:59.247: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 8.11491878s Jan 9 08:07:01.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 10.116531676s ... skipping 18 lines ... Jan 9 08:07:39.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 48.116851552s Jan 9 08:07:41.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 50.116056978s Jan 9 08:07:43.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Pending", Reason="", readiness=false. Elapsed: 52.116176641s Jan 9 08:07:45.247: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Running", Reason="", readiness=true. Elapsed: 54.114844048s Jan 9 08:07:47.249: INFO: Pod "azuredisk-volume-tester-7zjng": Phase="Succeeded", Reason="", readiness=false. Elapsed: 56.116223892s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:07:47.249[0m Jan 9 08:07:47.249: INFO: Pod "azuredisk-volume-tester-7zjng" satisfied condition "Succeeded or Failed" Jan 9 08:07:47.249: INFO: deleting Pod "azuredisk-4728"/"azuredisk-volume-tester-7zjng" Jan 9 08:07:47.343: INFO: Pod azuredisk-volume-tester-7zjng has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7zjng in namespace azuredisk-4728 [38;5;243m01/09/23 08:07:47.343[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:07:47.478[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:07:47.535[0m ... skipping 34 lines ... [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:08:29.229[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:08:29.23[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:08:29.288[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:08:29.289[0m [1mSTEP:[0m checking that the pod has 'FailedMount' event [38;5;243m01/09/23 08:08:29.348[0m Jan 9 08:08:51.466: INFO: deleting Pod "azuredisk-5466"/"azuredisk-volume-tester-t4lnj" Jan 9 08:08:51.527: INFO: Error getting logs for pod azuredisk-volume-tester-t4lnj: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-t4lnj) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-t4lnj in namespace azuredisk-5466 [38;5;243m01/09/23 08:08:51.527[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:08:51.642[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:08:51.698[0m Jan 9 08:08:51.698: INFO: deleting PVC "azuredisk-5466"/"pvc-9bx47" Jan 9 08:08:51.698: INFO: Deleting PersistentVolumeClaim "pvc-9bx47" [1mSTEP:[0m waiting for claim's PV "pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034" to be deleted [38;5;243m01/09/23 08:08:51.756[0m ... skipping 33 lines ... [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:08:29.229[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:08:29.23[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:08:29.288[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:08:29.289[0m [1mSTEP:[0m checking that the pod has 'FailedMount' event [38;5;243m01/09/23 08:08:29.348[0m Jan 9 08:08:51.466: INFO: deleting Pod "azuredisk-5466"/"azuredisk-volume-tester-t4lnj" Jan 9 08:08:51.527: INFO: Error getting logs for pod azuredisk-volume-tester-t4lnj: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-t4lnj) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-t4lnj in namespace azuredisk-5466 [38;5;243m01/09/23 08:08:51.527[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:08:51.642[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:08:51.698[0m Jan 9 08:08:51.698: INFO: deleting PVC "azuredisk-5466"/"pvc-9bx47" Jan 9 08:08:51.698: INFO: Deleting PersistentVolumeClaim "pvc-9bx47" [1mSTEP:[0m waiting for claim's PV "pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034" to be deleted [38;5;243m01/09/23 08:08:51.756[0m ... skipping 30 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:09:38.433[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:09:38.433[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:09:38.492[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:09:38.492[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:09:38.553[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:09:38.553[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:09:38.612[0m Jan 9 08:09:38.612: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-rbfr8" in namespace "azuredisk-2790" to be "Succeeded or Failed" Jan 9 08:09:38.668: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 56.249529ms Jan 9 08:09:40.726: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113388083s Jan 9 08:09:42.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114871577s Jan 9 08:09:44.728: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 6.115383545s Jan 9 08:09:46.728: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115907017s Jan 9 08:09:48.726: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114228576s ... skipping 2 lines ... Jan 9 08:09:54.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 16.114677858s Jan 9 08:09:56.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115200149s Jan 9 08:09:58.726: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 20.11420659s Jan 9 08:10:00.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 22.115182303s Jan 9 08:10:02.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.114737416s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:10:02.727[0m Jan 9 08:10:02.727: INFO: Pod "azuredisk-volume-tester-rbfr8" satisfied condition "Succeeded or Failed" Jan 9 08:10:02.727: INFO: deleting Pod "azuredisk-2790"/"azuredisk-volume-tester-rbfr8" Jan 9 08:10:02.790: INFO: Pod azuredisk-volume-tester-rbfr8 has the following logs: e2e-test [1mSTEP:[0m Deleting pod azuredisk-volume-tester-rbfr8 in namespace azuredisk-2790 [38;5;243m01/09/23 08:10:02.79[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:10:02.913[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:10:02.97[0m ... skipping 39 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:09:38.433[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:09:38.433[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:09:38.492[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:09:38.492[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:09:38.553[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:09:38.553[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:09:38.612[0m Jan 9 08:09:38.612: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-rbfr8" in namespace "azuredisk-2790" to be "Succeeded or Failed" Jan 9 08:09:38.668: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 56.249529ms Jan 9 08:09:40.726: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113388083s Jan 9 08:09:42.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114871577s Jan 9 08:09:44.728: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 6.115383545s Jan 9 08:09:46.728: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115907017s Jan 9 08:09:48.726: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114228576s ... skipping 2 lines ... Jan 9 08:09:54.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 16.114677858s Jan 9 08:09:56.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115200149s Jan 9 08:09:58.726: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 20.11420659s Jan 9 08:10:00.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Pending", Reason="", readiness=false. Elapsed: 22.115182303s Jan 9 08:10:02.727: INFO: Pod "azuredisk-volume-tester-rbfr8": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.114737416s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:10:02.727[0m Jan 9 08:10:02.727: INFO: Pod "azuredisk-volume-tester-rbfr8" satisfied condition "Succeeded or Failed" Jan 9 08:10:02.727: INFO: deleting Pod "azuredisk-2790"/"azuredisk-volume-tester-rbfr8" Jan 9 08:10:02.790: INFO: Pod azuredisk-volume-tester-rbfr8 has the following logs: e2e-test [1mSTEP:[0m Deleting pod azuredisk-volume-tester-rbfr8 in namespace azuredisk-2790 [38;5;243m01/09/23 08:10:02.79[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:10:02.913[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:10:02.97[0m ... skipping 43 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:11:16.312[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:11:16.313[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:11:16.313[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:11:16.373[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:11:16.373[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:11:16.433[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:11:16.493[0m Jan 9 08:11:16.493: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-kkkx4" in namespace "azuredisk-5356" to be "Succeeded or Failed" Jan 9 08:11:16.550: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 56.473295ms Jan 9 08:11:18.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114654419s Jan 9 08:11:20.609: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.115264078s Jan 9 08:11:22.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114807492s Jan 9 08:11:24.607: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.113484479s Jan 9 08:11:26.609: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115374873s ... skipping 2 lines ... Jan 9 08:11:32.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115054172s Jan 9 08:11:34.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114749701s Jan 9 08:11:36.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 20.114663447s Jan 9 08:11:38.607: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 22.114170276s Jan 9 08:11:40.607: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.114000734s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:11:40.607[0m Jan 9 08:11:40.608: INFO: Pod "azuredisk-volume-tester-kkkx4" satisfied condition "Succeeded or Failed" Jan 9 08:11:40.608: INFO: deleting Pod "azuredisk-5356"/"azuredisk-volume-tester-kkkx4" Jan 9 08:11:40.694: INFO: Pod azuredisk-volume-tester-kkkx4 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-kkkx4 in namespace azuredisk-5356 [38;5;243m01/09/23 08:11:40.694[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:11:40.817[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:11:40.874[0m ... skipping 37 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:11:16.312[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:11:16.313[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:11:16.313[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:11:16.373[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:11:16.373[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:11:16.433[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:11:16.493[0m Jan 9 08:11:16.493: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-kkkx4" in namespace "azuredisk-5356" to be "Succeeded or Failed" Jan 9 08:11:16.550: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 56.473295ms Jan 9 08:11:18.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114654419s Jan 9 08:11:20.609: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.115264078s Jan 9 08:11:22.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114807492s Jan 9 08:11:24.607: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.113484479s Jan 9 08:11:26.609: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115374873s ... skipping 2 lines ... Jan 9 08:11:32.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115054172s Jan 9 08:11:34.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114749701s Jan 9 08:11:36.608: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 20.114663447s Jan 9 08:11:38.607: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Pending", Reason="", readiness=false. Elapsed: 22.114170276s Jan 9 08:11:40.607: INFO: Pod "azuredisk-volume-tester-kkkx4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.114000734s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:11:40.607[0m Jan 9 08:11:40.608: INFO: Pod "azuredisk-volume-tester-kkkx4" satisfied condition "Succeeded or Failed" Jan 9 08:11:40.608: INFO: deleting Pod "azuredisk-5356"/"azuredisk-volume-tester-kkkx4" Jan 9 08:11:40.694: INFO: Pod azuredisk-volume-tester-kkkx4 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-kkkx4 in namespace azuredisk-5356 [38;5;243m01/09/23 08:11:40.694[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:11:40.817[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:11:40.874[0m ... skipping 44 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:12:39.948[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:12:39.948[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:12:39.948[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:12:40.006[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:12:40.006[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:12:40.064[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:12:40.122[0m Jan 9 08:12:40.122: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-v2dvj" in namespace "azuredisk-5194" to be "Succeeded or Failed" Jan 9 08:12:40.178: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 56.505859ms Jan 9 08:12:42.236: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114665491s Jan 9 08:12:44.236: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114055902s Jan 9 08:12:46.236: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 6.113692649s Jan 9 08:12:48.238: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 8.116127856s Jan 9 08:12:50.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11493333s ... skipping 9 lines ... Jan 9 08:13:10.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 30.115573455s Jan 9 08:13:12.238: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 32.116014366s Jan 9 08:13:14.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 34.114979604s Jan 9 08:13:16.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 36.115140818s Jan 9 08:13:18.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.114827861s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:13:18.237[0m Jan 9 08:13:18.237: INFO: Pod "azuredisk-volume-tester-v2dvj" satisfied condition "Succeeded or Failed" Jan 9 08:13:18.237: INFO: deleting Pod "azuredisk-5194"/"azuredisk-volume-tester-v2dvj" Jan 9 08:13:18.324: INFO: Pod azuredisk-volume-tester-v2dvj has the following logs: hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-v2dvj in namespace azuredisk-5194 [38;5;243m01/09/23 08:13:18.324[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:13:18.445[0m ... skipping 63 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:12:39.948[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:12:39.948[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:12:39.948[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:12:40.006[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:12:40.006[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:12:40.064[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:12:40.122[0m Jan 9 08:12:40.122: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-v2dvj" in namespace "azuredisk-5194" to be "Succeeded or Failed" Jan 9 08:12:40.178: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 56.505859ms Jan 9 08:12:42.236: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114665491s Jan 9 08:12:44.236: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114055902s Jan 9 08:12:46.236: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 6.113692649s Jan 9 08:12:48.238: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 8.116127856s Jan 9 08:12:50.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11493333s ... skipping 9 lines ... Jan 9 08:13:10.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 30.115573455s Jan 9 08:13:12.238: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 32.116014366s Jan 9 08:13:14.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 34.114979604s Jan 9 08:13:16.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Pending", Reason="", readiness=false. Elapsed: 36.115140818s Jan 9 08:13:18.237: INFO: Pod "azuredisk-volume-tester-v2dvj": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.114827861s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:13:18.237[0m Jan 9 08:13:18.237: INFO: Pod "azuredisk-volume-tester-v2dvj" satisfied condition "Succeeded or Failed" Jan 9 08:13:18.237: INFO: deleting Pod "azuredisk-5194"/"azuredisk-volume-tester-v2dvj" Jan 9 08:13:18.324: INFO: Pod azuredisk-volume-tester-v2dvj has the following logs: hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-v2dvj in namespace azuredisk-5194 [38;5;243m01/09/23 08:13:18.324[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:13:18.445[0m ... skipping 53 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:16:14.32[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:16:14.32[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:16:14.376[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:16:14.376[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:16:14.434[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:16:14.435[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/09/23 08:16:14.491[0m Jan 9 08:16:14.491: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-vt6bm" in namespace "azuredisk-1353" to be "Error status code" Jan 9 08:16:14.545: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 53.890416ms Jan 9 08:16:16.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.111597784s Jan 9 08:16:18.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111318037s Jan 9 08:16:20.604: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.112719346s Jan 9 08:16:22.603: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111987763s Jan 9 08:16:24.600: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109364628s ... skipping 24 lines ... Jan 9 08:17:14.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.110598567s Jan 9 08:17:16.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.111019771s Jan 9 08:17:18.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.109910756s Jan 9 08:17:20.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.111316554s Jan 9 08:17:22.599: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.108537679s Jan 9 08:17:24.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.110155149s Jan 9 08:17:26.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Failed", Reason="", readiness=false. Elapsed: 1m12.109975041s [1mSTEP:[0m Saw pod failure [38;5;243m01/09/23 08:17:26.601[0m Jan 9 08:17:26.601: INFO: Pod "azuredisk-volume-tester-vt6bm" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/09/23 08:17:26.601[0m Jan 9 08:17:26.691: INFO: deleting Pod "azuredisk-1353"/"azuredisk-volume-tester-vt6bm" Jan 9 08:17:26.749: INFO: Pod azuredisk-volume-tester-vt6bm has the following logs: touch: /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-vt6bm in namespace azuredisk-1353 [38;5;243m01/09/23 08:17:26.75[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:17:26.868[0m ... skipping 34 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:16:14.32[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:16:14.32[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:16:14.376[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:16:14.376[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:16:14.434[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:16:14.435[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/09/23 08:16:14.491[0m Jan 9 08:16:14.491: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-vt6bm" in namespace "azuredisk-1353" to be "Error status code" Jan 9 08:16:14.545: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 53.890416ms Jan 9 08:16:16.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.111597784s Jan 9 08:16:18.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111318037s Jan 9 08:16:20.604: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.112719346s Jan 9 08:16:22.603: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111987763s Jan 9 08:16:24.600: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109364628s ... skipping 24 lines ... Jan 9 08:17:14.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.110598567s Jan 9 08:17:16.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.111019771s Jan 9 08:17:18.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.109910756s Jan 9 08:17:20.602: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.111316554s Jan 9 08:17:22.599: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.108537679s Jan 9 08:17:24.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.110155149s Jan 9 08:17:26.601: INFO: Pod "azuredisk-volume-tester-vt6bm": Phase="Failed", Reason="", readiness=false. Elapsed: 1m12.109975041s [1mSTEP:[0m Saw pod failure [38;5;243m01/09/23 08:17:26.601[0m Jan 9 08:17:26.601: INFO: Pod "azuredisk-volume-tester-vt6bm" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/09/23 08:17:26.601[0m Jan 9 08:17:26.691: INFO: deleting Pod "azuredisk-1353"/"azuredisk-volume-tester-vt6bm" Jan 9 08:17:26.749: INFO: Pod azuredisk-volume-tester-vt6bm has the following logs: touch: /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-vt6bm in namespace azuredisk-1353 [38;5;243m01/09/23 08:17:26.75[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:17:26.868[0m ... skipping 663 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:25:49.425[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:25:49.425[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:25:49.481[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:25:49.481[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:25:49.542[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:25:49.542[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:25:49.598[0m Jan 9 08:25:49.598: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-569pv" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 9 08:25:49.653: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 54.244875ms Jan 9 08:25:51.708: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109629006s Jan 9 08:25:53.706: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.108027108s Jan 9 08:25:55.709: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110386661s Jan 9 08:25:57.709: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110210227s Jan 9 08:25:59.709: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11084133s ... skipping 2 lines ... Jan 9 08:26:05.707: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 16.109074049s Jan 9 08:26:07.710: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 18.111234546s Jan 9 08:26:09.710: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 20.111266461s Jan 9 08:26:11.707: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 22.108917931s Jan 9 08:26:13.708: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.109579711s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:26:13.708[0m Jan 9 08:26:13.708: INFO: Pod "azuredisk-volume-tester-569pv" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/09/23 08:26:13.708[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/09/23 08:26:18.711[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:26:18.82[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:26:18.821[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:26:18.88[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/09/23 08:26:18.88[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:26:18.937[0m Jan 9 08:26:18.937: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-zgvnr" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 9 08:26:18.991: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 54.306359ms Jan 9 08:26:21.048: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110628336s Jan 9 08:26:23.047: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110360845s Jan 9 08:26:25.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109015079s Jan 9 08:26:27.047: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110520175s Jan 9 08:26:29.048: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110746704s ... skipping 10 lines ... Jan 9 08:26:51.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 32.109595098s Jan 9 08:26:53.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 34.109418235s Jan 9 08:26:55.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 36.109486424s Jan 9 08:26:57.048: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 38.110985255s Jan 9 08:26:59.047: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.109637376s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:26:59.047[0m Jan 9 08:26:59.047: INFO: Pod "azuredisk-volume-tester-zgvnr" satisfied condition "Succeeded or Failed" Jan 9 08:26:59.047: INFO: deleting Pod "azuredisk-59"/"azuredisk-volume-tester-zgvnr" Jan 9 08:26:59.131: INFO: Pod azuredisk-volume-tester-zgvnr has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-zgvnr in namespace azuredisk-59 [38;5;243m01/09/23 08:26:59.131[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:26:59.248[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:26:59.303[0m ... skipping 47 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:25:49.425[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:25:49.425[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:25:49.481[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:25:49.481[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:25:49.542[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:25:49.542[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:25:49.598[0m Jan 9 08:25:49.598: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-569pv" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 9 08:25:49.653: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 54.244875ms Jan 9 08:25:51.708: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109629006s Jan 9 08:25:53.706: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.108027108s Jan 9 08:25:55.709: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110386661s Jan 9 08:25:57.709: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110210227s Jan 9 08:25:59.709: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11084133s ... skipping 2 lines ... Jan 9 08:26:05.707: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 16.109074049s Jan 9 08:26:07.710: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 18.111234546s Jan 9 08:26:09.710: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 20.111266461s Jan 9 08:26:11.707: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Pending", Reason="", readiness=false. Elapsed: 22.108917931s Jan 9 08:26:13.708: INFO: Pod "azuredisk-volume-tester-569pv": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.109579711s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:26:13.708[0m Jan 9 08:26:13.708: INFO: Pod "azuredisk-volume-tester-569pv" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/09/23 08:26:13.708[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/09/23 08:26:18.711[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:26:18.82[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:26:18.821[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:26:18.88[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/09/23 08:26:18.88[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:26:18.937[0m Jan 9 08:26:18.937: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-zgvnr" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 9 08:26:18.991: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 54.306359ms Jan 9 08:26:21.048: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110628336s Jan 9 08:26:23.047: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110360845s Jan 9 08:26:25.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109015079s Jan 9 08:26:27.047: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110520175s Jan 9 08:26:29.048: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110746704s ... skipping 10 lines ... Jan 9 08:26:51.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 32.109595098s Jan 9 08:26:53.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 34.109418235s Jan 9 08:26:55.046: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 36.109486424s Jan 9 08:26:57.048: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Pending", Reason="", readiness=false. Elapsed: 38.110985255s Jan 9 08:26:59.047: INFO: Pod "azuredisk-volume-tester-zgvnr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.109637376s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:26:59.047[0m Jan 9 08:26:59.047: INFO: Pod "azuredisk-volume-tester-zgvnr" satisfied condition "Succeeded or Failed" Jan 9 08:26:59.047: INFO: deleting Pod "azuredisk-59"/"azuredisk-volume-tester-zgvnr" Jan 9 08:26:59.131: INFO: Pod azuredisk-volume-tester-zgvnr has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-zgvnr in namespace azuredisk-59 [38;5;243m01/09/23 08:26:59.131[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:26:59.248[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:26:59.303[0m ... skipping 46 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:27:51.433[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:27:51.434[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:27:51.49[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:27:51.49[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:27:51.548[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:27:51.548[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:27:51.604[0m Jan 9 08:27:51.604: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-jflv5" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 9 08:27:51.658: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 54.637002ms Jan 9 08:27:53.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11099709s Jan 9 08:27:55.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111006732s Jan 9 08:27:57.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110621657s Jan 9 08:27:59.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111513217s Jan 9 08:28:01.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110730095s ... skipping 2 lines ... Jan 9 08:28:07.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 16.109924172s Jan 9 08:28:09.713: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 18.10970466s Jan 9 08:28:11.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 20.110569621s Jan 9 08:28:13.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 22.111607353s Jan 9 08:28:15.713: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.109624028s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:28:15.713[0m Jan 9 08:28:15.714: INFO: Pod "azuredisk-volume-tester-jflv5" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/09/23 08:28:15.714[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/09/23 08:28:20.714[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:28:20.825[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:28:20.825[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:28:20.883[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/09/23 08:28:20.883[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:28:20.943[0m Jan 9 08:28:20.943: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7w86q" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 9 08:28:20.998: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 54.358185ms Jan 9 08:28:23.054: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110266524s Jan 9 08:28:25.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.10929594s Jan 9 08:28:27.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109182516s Jan 9 08:28:29.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.109407476s Jan 9 08:28:31.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.10933314s ... skipping 2 lines ... Jan 9 08:28:37.052: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 16.108948834s Jan 9 08:28:39.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 18.10970886s Jan 9 08:28:41.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 20.109841785s Jan 9 08:28:43.052: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 22.10869874s Jan 9 08:28:45.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.109982419s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:28:45.054[0m Jan 9 08:28:45.054: INFO: Pod "azuredisk-volume-tester-7w86q" satisfied condition "Succeeded or Failed" Jan 9 08:28:45.054: INFO: deleting Pod "azuredisk-2546"/"azuredisk-volume-tester-7w86q" Jan 9 08:28:45.112: INFO: Pod azuredisk-volume-tester-7w86q has the following logs: 20.0G [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7w86q in namespace azuredisk-2546 [38;5;243m01/09/23 08:28:45.112[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:28:45.227[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:28:45.282[0m ... skipping 46 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:27:51.433[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:27:51.434[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:27:51.49[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:27:51.49[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:27:51.548[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:27:51.548[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:27:51.604[0m Jan 9 08:27:51.604: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-jflv5" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 9 08:27:51.658: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 54.637002ms Jan 9 08:27:53.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11099709s Jan 9 08:27:55.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111006732s Jan 9 08:27:57.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110621657s Jan 9 08:27:59.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111513217s Jan 9 08:28:01.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110730095s ... skipping 2 lines ... Jan 9 08:28:07.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 16.109924172s Jan 9 08:28:09.713: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 18.10970466s Jan 9 08:28:11.714: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 20.110569621s Jan 9 08:28:13.715: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Pending", Reason="", readiness=false. Elapsed: 22.111607353s Jan 9 08:28:15.713: INFO: Pod "azuredisk-volume-tester-jflv5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.109624028s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:28:15.713[0m Jan 9 08:28:15.714: INFO: Pod "azuredisk-volume-tester-jflv5" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/09/23 08:28:15.714[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/09/23 08:28:20.714[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:28:20.825[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:28:20.825[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:28:20.883[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/09/23 08:28:20.883[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:28:20.943[0m Jan 9 08:28:20.943: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7w86q" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 9 08:28:20.998: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 54.358185ms Jan 9 08:28:23.054: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110266524s Jan 9 08:28:25.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.10929594s Jan 9 08:28:27.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109182516s Jan 9 08:28:29.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.109407476s Jan 9 08:28:31.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.10933314s ... skipping 2 lines ... Jan 9 08:28:37.052: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 16.108948834s Jan 9 08:28:39.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 18.10970886s Jan 9 08:28:41.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 20.109841785s Jan 9 08:28:43.052: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Pending", Reason="", readiness=false. Elapsed: 22.10869874s Jan 9 08:28:45.053: INFO: Pod "azuredisk-volume-tester-7w86q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.109982419s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:28:45.054[0m Jan 9 08:28:45.054: INFO: Pod "azuredisk-volume-tester-7w86q" satisfied condition "Succeeded or Failed" Jan 9 08:28:45.054: INFO: deleting Pod "azuredisk-2546"/"azuredisk-volume-tester-7w86q" Jan 9 08:28:45.112: INFO: Pod azuredisk-volume-tester-7w86q has the following logs: 20.0G [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7w86q in namespace azuredisk-2546 [38;5;243m01/09/23 08:28:45.112[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:28:45.227[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:28:45.282[0m ... skipping 55 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:29:32.572[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:29:32.572[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:29:32.627[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:29:32.627[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:29:32.682[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:29:32.682[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:29:32.738[0m Jan 9 08:29:32.738: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-shp2h" in namespace "azuredisk-1598" to be "Succeeded or Failed" Jan 9 08:29:32.792: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 53.946381ms Jan 9 08:29:34.850: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 2.112343947s Jan 9 08:29:36.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 4.109004001s Jan 9 08:29:38.848: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109952813s Jan 9 08:29:40.849: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111163883s Jan 9 08:29:42.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109602677s ... skipping 9 lines ... Jan 9 08:30:02.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 30.109641943s Jan 9 08:30:04.849: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 32.110911157s Jan 9 08:30:06.852: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 34.113948585s Jan 9 08:30:08.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 36.109774568s Jan 9 08:30:10.848: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.110437875s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:30:10.848[0m Jan 9 08:30:10.848: INFO: Pod "azuredisk-volume-tester-shp2h" satisfied condition "Succeeded or Failed" Jan 9 08:30:10.848: INFO: deleting Pod "azuredisk-1598"/"azuredisk-volume-tester-shp2h" Jan 9 08:30:10.906: INFO: Pod azuredisk-volume-tester-shp2h has the following logs: hello world hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-shp2h in namespace azuredisk-1598 [38;5;243m01/09/23 08:30:10.906[0m ... skipping 70 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:29:32.572[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:29:32.572[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:29:32.627[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:29:32.627[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:29:32.682[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:29:32.682[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:29:32.738[0m Jan 9 08:29:32.738: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-shp2h" in namespace "azuredisk-1598" to be "Succeeded or Failed" Jan 9 08:29:32.792: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 53.946381ms Jan 9 08:29:34.850: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 2.112343947s Jan 9 08:29:36.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 4.109004001s Jan 9 08:29:38.848: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109952813s Jan 9 08:29:40.849: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111163883s Jan 9 08:29:42.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109602677s ... skipping 9 lines ... Jan 9 08:30:02.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 30.109641943s Jan 9 08:30:04.849: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 32.110911157s Jan 9 08:30:06.852: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 34.113948585s Jan 9 08:30:08.847: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Pending", Reason="", readiness=false. Elapsed: 36.109774568s Jan 9 08:30:10.848: INFO: Pod "azuredisk-volume-tester-shp2h": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.110437875s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:30:10.848[0m Jan 9 08:30:10.848: INFO: Pod "azuredisk-volume-tester-shp2h" satisfied condition "Succeeded or Failed" Jan 9 08:30:10.848: INFO: deleting Pod "azuredisk-1598"/"azuredisk-volume-tester-shp2h" Jan 9 08:30:10.906: INFO: Pod azuredisk-volume-tester-shp2h has the following logs: hello world hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-shp2h in namespace azuredisk-1598 [38;5;243m01/09/23 08:30:10.906[0m ... skipping 64 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:31:18.743[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:31:18.743[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:31:18.799[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:31:18.799[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:31:18.855[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:31:18.855[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:31:18.912[0m Jan 9 08:31:18.912: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-d8s5k" in namespace "azuredisk-3410" to be "Succeeded or Failed" Jan 9 08:31:18.966: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 54.124229ms Jan 9 08:31:21.023: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110662525s Jan 9 08:31:23.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 4.10883415s Jan 9 08:31:25.024: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 6.112351899s Jan 9 08:31:27.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 8.109494626s Jan 9 08:31:29.022: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110327225s ... skipping 9 lines ... Jan 9 08:31:49.023: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 30.111226489s Jan 9 08:31:51.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 32.108619276s Jan 9 08:31:53.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 34.109157138s Jan 9 08:31:55.022: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 36.110515006s Jan 9 08:31:57.022: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.110170949s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:31:57.022[0m Jan 9 08:31:57.022: INFO: Pod "azuredisk-volume-tester-d8s5k" satisfied condition "Succeeded or Failed" Jan 9 08:31:57.022: INFO: deleting Pod "azuredisk-3410"/"azuredisk-volume-tester-d8s5k" Jan 9 08:31:57.105: INFO: Pod azuredisk-volume-tester-d8s5k has the following logs: 100+0 records in 100+0 records out 104857600 bytes (100.0MB) copied, 0.074370 seconds, 1.3GB/s hello world ... skipping 59 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:31:18.743[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:31:18.743[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:31:18.799[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:31:18.799[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:31:18.855[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:31:18.855[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:31:18.912[0m Jan 9 08:31:18.912: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-d8s5k" in namespace "azuredisk-3410" to be "Succeeded or Failed" Jan 9 08:31:18.966: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 54.124229ms Jan 9 08:31:21.023: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110662525s Jan 9 08:31:23.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 4.10883415s Jan 9 08:31:25.024: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 6.112351899s Jan 9 08:31:27.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 8.109494626s Jan 9 08:31:29.022: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110327225s ... skipping 9 lines ... Jan 9 08:31:49.023: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 30.111226489s Jan 9 08:31:51.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 32.108619276s Jan 9 08:31:53.021: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 34.109157138s Jan 9 08:31:55.022: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Pending", Reason="", readiness=false. Elapsed: 36.110515006s Jan 9 08:31:57.022: INFO: Pod "azuredisk-volume-tester-d8s5k": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.110170949s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:31:57.022[0m Jan 9 08:31:57.022: INFO: Pod "azuredisk-volume-tester-d8s5k" satisfied condition "Succeeded or Failed" Jan 9 08:31:57.022: INFO: deleting Pod "azuredisk-3410"/"azuredisk-volume-tester-d8s5k" Jan 9 08:31:57.105: INFO: Pod azuredisk-volume-tester-d8s5k has the following logs: 100+0 records in 100+0 records out 104857600 bytes (100.0MB) copied, 0.074370 seconds, 1.3GB/s hello world ... skipping 52 lines ... Jan 9 08:33:19.651: INFO: >>> kubeConfig: /root/tmp2655200364/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:33:19.652[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:33:19.652[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:33:19.709[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:33:19.709[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:33:19.766[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:33:19.822[0m Jan 9 08:33:19.822: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-n2pv7" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 9 08:33:19.877: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 54.331336ms Jan 9 08:33:21.933: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110428552s Jan 9 08:33:23.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.109525741s Jan 9 08:33:25.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109762138s Jan 9 08:33:27.931: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.108856504s Jan 9 08:33:29.933: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110589136s ... skipping 2 lines ... Jan 9 08:33:35.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 16.109887588s Jan 9 08:33:37.931: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 18.108930287s Jan 9 08:33:39.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 20.109671726s Jan 9 08:33:41.935: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 22.112889185s Jan 9 08:33:43.933: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.110448978s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:33:43.933[0m Jan 9 08:33:43.933: INFO: Pod "azuredisk-volume-tester-n2pv7" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/09/23 08:33:43.933[0m 2023/01/09 08:33:43 Running in Prow, converting AZURE_CREDENTIALS to AZURE_CREDENTIAL_FILE 2023/01/09 08:33:43 Reading credentials file /etc/azure-cred/credentials [1mSTEP:[0m Prow test resource group: kubetest-y4vzlmmp [38;5;243m01/09/23 08:33:43.934[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-53d304bc-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:33:43.935[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-53d304bc-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:33:45.381[0m ... skipping 5 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:34:00.559[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:34:00.559[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:34:00.616[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:34:00.616[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:34:00.674[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/09/23 08:34:00.674[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:34:00.73[0m Jan 9 08:34:00.730: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-dlc57" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 9 08:34:00.784: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 53.917288ms Jan 9 08:34:02.840: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110018298s Jan 9 08:34:04.841: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110827662s Jan 9 08:34:06.838: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 6.108594837s Jan 9 08:34:08.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 8.108769048s Jan 9 08:34:10.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109079349s ... skipping 10 lines ... Jan 9 08:34:32.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 32.108982875s Jan 9 08:34:34.841: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 34.11106361s Jan 9 08:34:36.838: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 36.108137497s Jan 9 08:34:38.840: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 38.110501817s Jan 9 08:34:40.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.10965611s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:34:40.839[0m Jan 9 08:34:40.840: INFO: Pod "azuredisk-volume-tester-dlc57" satisfied condition "Succeeded or Failed" Jan 9 08:34:40.840: INFO: deleting Pod "azuredisk-8582"/"azuredisk-volume-tester-dlc57" Jan 9 08:34:40.937: INFO: Pod azuredisk-volume-tester-dlc57 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-dlc57 in namespace azuredisk-8582 [38;5;243m01/09/23 08:34:40.937[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:34:41.054[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:34:41.109[0m ... skipping 50 lines ... Jan 9 08:33:19.651: INFO: >>> kubeConfig: /root/tmp2655200364/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:33:19.652[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:33:19.652[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:33:19.709[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:33:19.709[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:33:19.766[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:33:19.822[0m Jan 9 08:33:19.822: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-n2pv7" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 9 08:33:19.877: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 54.331336ms Jan 9 08:33:21.933: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110428552s Jan 9 08:33:23.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.109525741s Jan 9 08:33:25.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109762138s Jan 9 08:33:27.931: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.108856504s Jan 9 08:33:29.933: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110589136s ... skipping 2 lines ... Jan 9 08:33:35.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 16.109887588s Jan 9 08:33:37.931: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 18.108930287s Jan 9 08:33:39.932: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 20.109671726s Jan 9 08:33:41.935: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Pending", Reason="", readiness=false. Elapsed: 22.112889185s Jan 9 08:33:43.933: INFO: Pod "azuredisk-volume-tester-n2pv7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.110448978s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:33:43.933[0m Jan 9 08:33:43.933: INFO: Pod "azuredisk-volume-tester-n2pv7" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/09/23 08:33:43.933[0m [1mSTEP:[0m Prow test resource group: kubetest-y4vzlmmp [38;5;243m01/09/23 08:33:43.934[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-53d304bc-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:33:43.935[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-53d304bc-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:33:45.381[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/09/23 08:33:45.381[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/09/23 08:33:45.381[0m ... skipping 3 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:34:00.559[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:34:00.559[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:34:00.616[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:34:00.616[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:34:00.674[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/09/23 08:34:00.674[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:34:00.73[0m Jan 9 08:34:00.730: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-dlc57" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 9 08:34:00.784: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 53.917288ms Jan 9 08:34:02.840: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110018298s Jan 9 08:34:04.841: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110827662s Jan 9 08:34:06.838: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 6.108594837s Jan 9 08:34:08.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 8.108769048s Jan 9 08:34:10.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109079349s ... skipping 10 lines ... Jan 9 08:34:32.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 32.108982875s Jan 9 08:34:34.841: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 34.11106361s Jan 9 08:34:36.838: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 36.108137497s Jan 9 08:34:38.840: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Pending", Reason="", readiness=false. Elapsed: 38.110501817s Jan 9 08:34:40.839: INFO: Pod "azuredisk-volume-tester-dlc57": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.10965611s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:34:40.839[0m Jan 9 08:34:40.840: INFO: Pod "azuredisk-volume-tester-dlc57" satisfied condition "Succeeded or Failed" Jan 9 08:34:40.840: INFO: deleting Pod "azuredisk-8582"/"azuredisk-volume-tester-dlc57" Jan 9 08:34:40.937: INFO: Pod azuredisk-volume-tester-dlc57 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-dlc57 in namespace azuredisk-8582 [38;5;243m01/09/23 08:34:40.937[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:34:41.054[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:34:41.109[0m ... skipping 49 lines ... Jan 9 08:36:50.488: INFO: >>> kubeConfig: /root/tmp2655200364/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:36:50.489[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:36:50.489[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:36:50.546[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:36:50.546[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:36:50.604[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:36:50.66[0m Jan 9 08:36:50.661: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-l7frb" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 9 08:36:50.714: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 53.721891ms Jan 9 08:36:52.770: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109601418s Jan 9 08:36:54.772: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111720627s Jan 9 08:36:56.771: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.1108049s Jan 9 08:36:58.769: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.108623458s Jan 9 08:37:00.770: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109094894s ... skipping 2 lines ... Jan 9 08:37:06.773: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 16.112144375s Jan 9 08:37:08.770: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 18.109623597s Jan 9 08:37:10.771: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 20.110848073s Jan 9 08:37:12.772: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 22.111311975s Jan 9 08:37:14.769: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.108723321s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:37:14.769[0m Jan 9 08:37:14.770: INFO: Pod "azuredisk-volume-tester-l7frb" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/09/23 08:37:14.77[0m 2023/01/09 08:37:14 Running in Prow, converting AZURE_CREDENTIALS to AZURE_CREDENTIAL_FILE 2023/01/09 08:37:14 Reading credentials file /etc/azure-cred/credentials [1mSTEP:[0m Prow test resource group: kubetest-y4vzlmmp [38;5;243m01/09/23 08:37:14.771[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-d17e0cd6-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:37:14.771[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-d17e0cd6-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:37:15.521[0m ... skipping 12 lines ... [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:37:32.859[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:37:32.916[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:37:32.916[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:37:32.975[0m [1mSTEP:[0m Set pod anti-affinity to make sure two pods are scheduled on different nodes [38;5;243m01/09/23 08:37:32.975[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/09/23 08:37:32.975[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:37:33.036[0m Jan 9 08:37:33.036: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-p6w4s" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 9 08:37:33.090: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 54.22978ms Jan 9 08:37:35.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110176688s Jan 9 08:37:37.147: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111373485s Jan 9 08:37:39.145: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109055781s Jan 9 08:37:41.147: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111244891s Jan 9 08:37:43.147: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110460651s Jan 9 08:37:45.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 12.110029465s Jan 9 08:37:47.155: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 14.119050578s Jan 9 08:37:49.158: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 16.121449219s Jan 9 08:37:51.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 18.109958359s Jan 9 08:37:53.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 20.109454766s Jan 9 08:37:55.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 22.109606512s Jan 9 08:37:57.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Failed", Reason="", readiness=false. Elapsed: 24.109761622s Jan 9 08:37:57.150: INFO: Unexpected error: <*fmt.wrapError | 0xc000726100>: { msg: "error while waiting for pod azuredisk-7726/azuredisk-volume-tester-p6w4s to be Succeeded or Failed: pod \"azuredisk-volume-tester-p6w4s\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.248.0.4 PodIP:10.248.0.22 PodIPs:[{IP:10.248.0.22}] StartTime:2023-01-09 08:37:36 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:2,Signal:0,Reason:Error,Message:,StartedAt:2023-01-09 08:37:54 +0000 UTC,FinishedAt:2023-01-09 08:37:54 +0000 UTC,ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce Started:0xc000c9031f}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", err: <*errors.errorString | 0xc000115b70>{ s: "pod \"azuredisk-volume-tester-p6w4s\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.248.0.4 PodIP:10.248.0.22 PodIPs:[{IP:10.248.0.22}] StartTime:2023-01-09 08:37:36 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:2,Signal:0,Reason:Error,Message:,StartedAt:2023-01-09 08:37:54 +0000 UTC,FinishedAt:2023-01-09 08:37:54 +0000 UTC,ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce Started:0xc000c9031f}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", }, } Jan 9 08:37:57.150: FAIL: error while waiting for pod azuredisk-7726/azuredisk-volume-tester-p6w4s to be Succeeded or Failed: pod "azuredisk-volume-tester-p6w4s" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.248.0.4 PodIP:10.248.0.22 PodIPs:[{IP:10.248.0.22}] StartTime:2023-01-09 08:37:36 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:2,Signal:0,Reason:Error,Message:,StartedAt:2023-01-09 08:37:54 +0000 UTC,FinishedAt:2023-01-09 08:37:54 +0000 UTC,ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce Started:0xc000c9031f}] QOSClass:BestEffort EphemeralContainerStatuses:[]} Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x225059f?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeSnapshotTest).Run(0xc000d51d78, {0x270a860, 0xc000ea61a0}, {0x26f5a60, 0xc000e3e0a0}, 0xc000a90dc0?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_snapshot_tester.go:142 +0x1358 ... skipping 42 lines ... Jan 9 08:40:05.579: INFO: Waiting up to 5m0s for PersistentVolumeClaim azuredisk-7726 to be removed Jan 9 08:40:05.632: INFO: Claim "azuredisk-7726" in namespace "pvc-wwqwv" doesn't exist in the system Jan 9 08:40:05.632: INFO: deleting StorageClass azuredisk-7726-disk.csi.azure.com-dynamic-sc-g9q89 [1mSTEP:[0m dump namespace information after failure [38;5;243m01/09/23 08:40:05.69[0m [1mSTEP:[0m Destroying namespace "azuredisk-7726" for this suite. [38;5;243m01/09/23 08:40:05.691[0m [38;5;243m------------------------------[0m [38;5;9m• [FAILED] [196.095 seconds][0m Dynamic Provisioning [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:41[0m [multi-az] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:48[0m [38;5;9m[1m[It] should create a pod, write to its pv, take a volume snapshot, overwrite data in original pv, create another pod from the snapshot, and read unaltered original data from original pv[disk.csi.azure.com][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:747[0m ... skipping 7 lines ... Jan 9 08:36:50.488: INFO: >>> kubeConfig: /root/tmp2655200364/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:36:50.489[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:36:50.489[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:36:50.546[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:36:50.546[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:36:50.604[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:36:50.66[0m Jan 9 08:36:50.661: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-l7frb" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 9 08:36:50.714: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 53.721891ms Jan 9 08:36:52.770: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109601418s Jan 9 08:36:54.772: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111720627s Jan 9 08:36:56.771: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.1108049s Jan 9 08:36:58.769: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.108623458s Jan 9 08:37:00.770: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109094894s ... skipping 2 lines ... Jan 9 08:37:06.773: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 16.112144375s Jan 9 08:37:08.770: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 18.109623597s Jan 9 08:37:10.771: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 20.110848073s Jan 9 08:37:12.772: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Pending", Reason="", readiness=false. Elapsed: 22.111311975s Jan 9 08:37:14.769: INFO: Pod "azuredisk-volume-tester-l7frb": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.108723321s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:37:14.769[0m Jan 9 08:37:14.770: INFO: Pod "azuredisk-volume-tester-l7frb" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/09/23 08:37:14.77[0m [1mSTEP:[0m Prow test resource group: kubetest-y4vzlmmp [38;5;243m01/09/23 08:37:14.771[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-d17e0cd6-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:37:14.771[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-d17e0cd6-8ff8-11ed-b8dd-5e7dfbe78601 [38;5;243m01/09/23 08:37:15.521[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/09/23 08:37:15.521[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/09/23 08:37:15.521[0m ... skipping 10 lines ... [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:37:32.859[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:37:32.916[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:37:32.916[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:37:32.975[0m [1mSTEP:[0m Set pod anti-affinity to make sure two pods are scheduled on different nodes [38;5;243m01/09/23 08:37:32.975[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/09/23 08:37:32.975[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:37:33.036[0m Jan 9 08:37:33.036: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-p6w4s" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 9 08:37:33.090: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 54.22978ms Jan 9 08:37:35.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110176688s Jan 9 08:37:37.147: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111373485s Jan 9 08:37:39.145: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 6.109055781s Jan 9 08:37:41.147: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111244891s Jan 9 08:37:43.147: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110460651s Jan 9 08:37:45.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 12.110029465s Jan 9 08:37:47.155: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 14.119050578s Jan 9 08:37:49.158: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 16.121449219s Jan 9 08:37:51.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 18.109958359s Jan 9 08:37:53.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 20.109454766s Jan 9 08:37:55.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Pending", Reason="", readiness=false. Elapsed: 22.109606512s Jan 9 08:37:57.146: INFO: Pod "azuredisk-volume-tester-p6w4s": Phase="Failed", Reason="", readiness=false. Elapsed: 24.109761622s Jan 9 08:37:57.150: INFO: Unexpected error: <*fmt.wrapError | 0xc000726100>: { msg: "error while waiting for pod azuredisk-7726/azuredisk-volume-tester-p6w4s to be Succeeded or Failed: pod \"azuredisk-volume-tester-p6w4s\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.248.0.4 PodIP:10.248.0.22 PodIPs:[{IP:10.248.0.22}] StartTime:2023-01-09 08:37:36 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:2,Signal:0,Reason:Error,Message:,StartedAt:2023-01-09 08:37:54 +0000 UTC,FinishedAt:2023-01-09 08:37:54 +0000 UTC,ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce Started:0xc000c9031f}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", err: <*errors.errorString | 0xc000115b70>{ s: "pod \"azuredisk-volume-tester-p6w4s\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.248.0.4 PodIP:10.248.0.22 PodIPs:[{IP:10.248.0.22}] StartTime:2023-01-09 08:37:36 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:2,Signal:0,Reason:Error,Message:,StartedAt:2023-01-09 08:37:54 +0000 UTC,FinishedAt:2023-01-09 08:37:54 +0000 UTC,ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce Started:0xc000c9031f}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", }, } Jan 9 08:37:57.150: FAIL: error while waiting for pod azuredisk-7726/azuredisk-volume-tester-p6w4s to be Succeeded or Failed: pod "azuredisk-volume-tester-p6w4s" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.248.0.4 PodIP:10.248.0.22 PodIPs:[{IP:10.248.0.22}] StartTime:2023-01-09 08:37:36 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:2,Signal:0,Reason:Error,Message:,StartedAt:2023-01-09 08:37:54 +0000 UTC,FinishedAt:2023-01-09 08:37:54 +0000 UTC,ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce Started:0xc000c9031f}] QOSClass:BestEffort EphemeralContainerStatuses:[]} Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x225059f?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeSnapshotTest).Run(0xc000d51d78, {0x270a860, 0xc000ea61a0}, {0x26f5a60, 0xc000e3e0a0}, 0xc000a90dc0?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_snapshot_tester.go:142 +0x1358 ... skipping 43 lines ... Jan 9 08:40:05.632: INFO: Claim "azuredisk-7726" in namespace "pvc-wwqwv" doesn't exist in the system Jan 9 08:40:05.632: INFO: deleting StorageClass azuredisk-7726-disk.csi.azure.com-dynamic-sc-g9q89 [1mSTEP:[0m dump namespace information after failure [38;5;243m01/09/23 08:40:05.69[0m [1mSTEP:[0m Destroying namespace "azuredisk-7726" for this suite. [38;5;243m01/09/23 08:40:05.691[0m [38;5;243m<< End Captured GinkgoWriter Output[0m [38;5;9mJan 9 08:37:57.150: error while waiting for pod azuredisk-7726/azuredisk-volume-tester-p6w4s to be Succeeded or Failed: pod "azuredisk-volume-tester-p6w4s" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-09 08:37:36 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.248.0.4 PodIP:10.248.0.22 PodIPs:[{IP:10.248.0.22}] StartTime:2023-01-09 08:37:36 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:2,Signal:0,Reason:Error,Message:,StartedAt:2023-01-09 08:37:54 +0000 UTC,FinishedAt:2023-01-09 08:37:54 +0000 UTC,ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://52b66d8aa62faf55f52a08595d61ac7bde8471812c51ce2d29c913a1457c44ce Started:0xc000c9031f}] QOSClass:BestEffort EphemeralContainerStatuses:[]}[0m [38;5;9mIn [1m[It][0m[38;5;9m at: [1m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [1mThere were additional failures detected after the initial failure:[0m [38;5;13m[PANICKED][0m [38;5;13mTest Panicked[0m [38;5;13mIn [1m[DeferCleanup (Each)][0m[38;5;13m at: [1m/usr/local/go/src/runtime/panic.go:260[0m [38;5;13mruntime error: invalid memory address or nil pointer dereference[0m [38;5;13mFull Stack Trace[0m k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:274 +0x5c k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0000343c0) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:271 +0x179 ... skipping 25 lines ... [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:40:06.787[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:40:06.841[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:40:06.841[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:40:06.897[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:40:06.897[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:40:06.953[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:40:07.009[0m Jan 9 08:40:07.009: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-2mcjr" in namespace "azuredisk-3086" to be "Succeeded or Failed" Jan 9 08:40:07.063: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 54.395044ms Jan 9 08:40:09.121: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.111912232s Jan 9 08:40:11.118: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.108880899s Jan 9 08:40:13.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110834952s Jan 9 08:40:15.119: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110103002s Jan 9 08:40:17.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11135834s ... skipping 9 lines ... Jan 9 08:40:37.118: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 30.109166233s Jan 9 08:40:39.121: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 32.111597448s Jan 9 08:40:41.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 34.110517238s Jan 9 08:40:43.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 36.111254059s Jan 9 08:40:45.119: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.109499259s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:40:45.119[0m Jan 9 08:40:45.119: INFO: Pod "azuredisk-volume-tester-2mcjr" satisfied condition "Succeeded or Failed" Jan 9 08:40:45.119: INFO: deleting Pod "azuredisk-3086"/"azuredisk-volume-tester-2mcjr" Jan 9 08:40:45.176: INFO: Pod azuredisk-volume-tester-2mcjr has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-2mcjr in namespace azuredisk-3086 [38;5;243m01/09/23 08:40:45.176[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:40:45.293[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:40:45.347[0m ... skipping 70 lines ... [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:40:06.787[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:40:06.841[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:40:06.841[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:40:06.897[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:40:06.897[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:40:06.953[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:40:07.009[0m Jan 9 08:40:07.009: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-2mcjr" in namespace "azuredisk-3086" to be "Succeeded or Failed" Jan 9 08:40:07.063: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 54.395044ms Jan 9 08:40:09.121: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.111912232s Jan 9 08:40:11.118: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.108880899s Jan 9 08:40:13.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110834952s Jan 9 08:40:15.119: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110103002s Jan 9 08:40:17.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11135834s ... skipping 9 lines ... Jan 9 08:40:37.118: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 30.109166233s Jan 9 08:40:39.121: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 32.111597448s Jan 9 08:40:41.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 34.110517238s Jan 9 08:40:43.120: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Pending", Reason="", readiness=false. Elapsed: 36.111254059s Jan 9 08:40:45.119: INFO: Pod "azuredisk-volume-tester-2mcjr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.109499259s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:40:45.119[0m Jan 9 08:40:45.119: INFO: Pod "azuredisk-volume-tester-2mcjr" satisfied condition "Succeeded or Failed" Jan 9 08:40:45.119: INFO: deleting Pod "azuredisk-3086"/"azuredisk-volume-tester-2mcjr" Jan 9 08:40:45.176: INFO: Pod azuredisk-volume-tester-2mcjr has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-2mcjr in namespace azuredisk-3086 [38;5;243m01/09/23 08:40:45.176[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:40:45.293[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:40:45.347[0m ... skipping 1026 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:55:43.266[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:55:43.266[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:55:43.325[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:55:43.325[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:55:43.386[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:55:43.386[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:55:43.445[0m Jan 9 08:55:43.445: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-brrbm" in namespace "azuredisk-1092" to be "Succeeded or Failed" Jan 9 08:55:43.502: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 57.039727ms Jan 9 08:55:45.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114842023s Jan 9 08:55:47.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114501276s Jan 9 08:55:49.559: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114328925s Jan 9 08:55:51.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115122627s Jan 9 08:55:53.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114777653s ... skipping 2 lines ... Jan 9 08:55:59.558: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 16.113354389s Jan 9 08:56:01.559: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114072991s Jan 9 08:56:03.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 20.114388515s Jan 9 08:56:05.561: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 22.116350442s Jan 9 08:56:07.559: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.113960684s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:56:07.559[0m Jan 9 08:56:07.559: INFO: Pod "azuredisk-volume-tester-brrbm" satisfied condition "Succeeded or Failed" Jan 9 08:56:07.559: INFO: deleting Pod "azuredisk-1092"/"azuredisk-volume-tester-brrbm" Jan 9 08:56:07.650: INFO: Pod azuredisk-volume-tester-brrbm has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-brrbm in namespace azuredisk-1092 [38;5;243m01/09/23 08:56:07.65[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:56:07.773[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:56:07.829[0m ... skipping 33 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/09/23 08:55:43.266[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/09/23 08:55:43.266[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/09/23 08:55:43.325[0m [1mSTEP:[0m creating a PVC [38;5;243m01/09/23 08:55:43.325[0m [1mSTEP:[0m setting up the pod [38;5;243m01/09/23 08:55:43.386[0m [1mSTEP:[0m deploying the pod [38;5;243m01/09/23 08:55:43.386[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/09/23 08:55:43.445[0m Jan 9 08:55:43.445: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-brrbm" in namespace "azuredisk-1092" to be "Succeeded or Failed" Jan 9 08:55:43.502: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 57.039727ms Jan 9 08:55:45.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114842023s Jan 9 08:55:47.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114501276s Jan 9 08:55:49.559: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114328925s Jan 9 08:55:51.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115122627s Jan 9 08:55:53.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114777653s ... skipping 2 lines ... Jan 9 08:55:59.558: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 16.113354389s Jan 9 08:56:01.559: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114072991s Jan 9 08:56:03.560: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 20.114388515s Jan 9 08:56:05.561: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Pending", Reason="", readiness=false. Elapsed: 22.116350442s Jan 9 08:56:07.559: INFO: Pod "azuredisk-volume-tester-brrbm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.113960684s [1mSTEP:[0m Saw pod success [38;5;243m01/09/23 08:56:07.559[0m Jan 9 08:56:07.559: INFO: Pod "azuredisk-volume-tester-brrbm" satisfied condition "Succeeded or Failed" Jan 9 08:56:07.559: INFO: deleting Pod "azuredisk-1092"/"azuredisk-volume-tester-brrbm" Jan 9 08:56:07.650: INFO: Pod azuredisk-volume-tester-brrbm has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-brrbm in namespace azuredisk-1092 [38;5;243m01/09/23 08:56:07.65[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/09/23 08:56:07.773[0m [1mSTEP:[0m checking the PV [38;5;243m01/09/23 08:56:07.829[0m ... skipping 95 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0109 08:04:21.059741 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 e2e-test I0109 08:04:21.060357 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0109 08:04:21.087353 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0109 08:04:21.087373 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0109 08:04:21.087382 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0109 08:04:21.087409 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0109 08:04:21.088212 1 azure_auth.go:245] Using AzurePublicCloud environment I0109 08:04:21.088260 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I0109 08:04:21.088286 1 azure.go:777] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 25 lines ... I0109 08:04:21.088634 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0109 08:04:21.088652 1 azure_vmasclient.go:70] Azure AvailabilitySetsClient (read ops) using rate limit config: QPS=6, bucket=20 I0109 08:04:21.088661 1 azure_vmasclient.go:73] Azure AvailabilitySetsClient (write ops) using rate limit config: QPS=100, bucket=1000 I0109 08:04:21.088738 1 azure.go:1008] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0109 08:04:21.088758 1 azuredisk.go:192] disable UseInstanceMetadata for controller I0109 08:04:21.088768 1 azuredisk.go:204] cloud: AzurePublicCloud, location: westus2, rg: kubetest-y4vzlmmp, VMType: vmss, PrimaryScaleSetName: k8s-agentpool-33023206-vmss, PrimaryAvailabilitySetName: , DisableAvailabilitySetNodes: false I0109 08:04:21.092227 1 mount_linux.go:287] 'umount /tmp/kubelet-detect-safe-umount1364018212' failed with: exit status 32, output: umount: /tmp/kubelet-detect-safe-umount1364018212: must be superuser to unmount. I0109 08:04:21.092259 1 mount_linux.go:289] Detected umount with unsafe 'not mounted' behavior I0109 08:04:21.092328 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0109 08:04:21.092338 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0109 08:04:21.092345 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0109 08:04:21.092352 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0109 08:04:21.092358 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 68 lines ... I0109 08:04:32.684647 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 24988 I0109 08:04:32.779950 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 32356 I0109 08:04:32.783126 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:04:32.783172 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 to node k8s-agentpool-33023206-vmss000001 I0109 08:04:32.783240 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8:%!s(*provider.AttachDiskOptions=&{None pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 false 0})] I0109 08:04:32.783304 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8:%!s(*provider.AttachDiskOptions=&{None pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 false 0})]) I0109 08:04:33.715644 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8:%!s(*provider.AttachDiskOptions=&{None pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:04:43.877161 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:04:43.877197 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:04:43.877218 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:04:43.877234 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:04:43.877281 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=11.281566128 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:04:43.877305 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 18 lines ... I0109 08:05:39.206146 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8) returned with <nil> I0109 08:05:39.206201 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.208348875 request="azuredisk_csi_driver_controller_delete_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-8f537237-c715-4dd5-9609-c7d10bd9a8b8" result_code="succeeded" I0109 08:05:39.206257 1 utils.go:84] GRPC response: {} I0109 08:05:44.691035 1 utils.go:77] GRPC call: /csi.v1.Controller/CreateVolume I0109 08:05:44.691058 1 utils.go:78] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-1","topology.kubernetes.io/zone":"westus2-1"}},{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-1","topology.kubernetes.io/zone":"westus2-1"}},{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2","csi.storage.k8s.io/pvc/name":"pvc-jw64p","csi.storage.k8s.io/pvc/namespace":"azuredisk-2540","enableAsyncAttach":"false","networkAccessPolicy":"DenyAll","skuName":"Standard_LRS","userAgent":"azuredisk-e2e-test"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]} I0109 08:05:44.691770 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0109 08:05:44.697733 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0109 08:05:44.697755 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0109 08:05:44.697763 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0109 08:05:44.697792 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0109 08:05:44.698130 1 azure_auth.go:245] Using AzurePublicCloud environment I0109 08:05:44.698176 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I0109 08:05:44.698203 1 azure.go:777] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 37 lines ... I0109 08:05:49.075114 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000000","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2","csi.storage.k8s.io/pvc/name":"pvc-jw64p","csi.storage.k8s.io/pvc/namespace":"azuredisk-2540","enableAsyncAttach":"false","enableasyncattach":"false","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com","userAgent":"azuredisk-e2e-test"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2"} I0109 08:05:49.099221 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1218 I0109 08:05:49.099497 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 to node k8s-agentpool-33023206-vmss000000 (vmState Succeeded). I0109 08:05:49.099525 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 to node k8s-agentpool-33023206-vmss000000 I0109 08:05:49.099561 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 lun 0 to node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 false 0})] I0109 08:05:49.099598 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 false 0})]) I0109 08:05:49.273431 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:05:59.388339 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000000) successfully I0109 08:05:59.388376 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000000) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:05:59.388396 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 attached to node k8s-agentpool-33023206-vmss000000. I0109 08:05:59.388410 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2 to node k8s-agentpool-33023206-vmss000000 successfully I0109 08:05:59.388455 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.288955769 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2" node="k8s-agentpool-33023206-vmss000000" result_code="succeeded" I0109 08:05:59.388474 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 18 lines ... I0109 08:06:45.640289 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2) returned with <nil> I0109 08:06:45.640326 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.255619419 request="azuredisk_csi_driver_controller_delete_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c650d3a2-9efa-4b74-902a-3f298a1385a2" result_code="succeeded" I0109 08:06:45.640343 1 utils.go:84] GRPC response: {} I0109 08:06:51.119854 1 utils.go:77] GRPC call: /csi.v1.Controller/CreateVolume I0109 08:06:51.119874 1 utils.go:78] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}}]},"capacity_range":{"required_bytes":1099511627776},"name":"pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331","parameters":{"csi.storage.k8s.io/pv/name":"pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331","csi.storage.k8s.io/pvc/name":"pvc-wpr95","csi.storage.k8s.io/pvc/namespace":"azuredisk-4728","enableAsyncAttach":"false","enableBursting":"true","perfProfile":"Basic","skuName":"Premium_LRS","userAgent":"azuredisk-e2e-test"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]} I0109 08:06:51.120503 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0109 08:06:51.130380 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0109 08:06:51.130402 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0109 08:06:51.130411 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0109 08:06:51.130440 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0109 08:06:51.131000 1 azure_auth.go:245] Using AzurePublicCloud environment I0109 08:06:51.131060 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I0109 08:06:51.131075 1 azure.go:777] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 37 lines ... I0109 08:06:54.158318 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331","csi.storage.k8s.io/pvc/name":"pvc-wpr95","csi.storage.k8s.io/pvc/namespace":"azuredisk-4728","enableAsyncAttach":"false","enableBursting":"true","enableasyncattach":"false","perfProfile":"Basic","requestedsizegib":"1024","skuName":"Premium_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com","userAgent":"azuredisk-e2e-test"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331"} I0109 08:06:54.239150 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1338 I0109 08:06:54.239665 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:06:54.239697 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 to node k8s-agentpool-33023206-vmss000001 I0109 08:06:54.239756 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 false 0})] I0109 08:06:54.239961 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 false 0})]) I0109 08:06:54.390111 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:07:04.607593 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:07:04.607627 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:07:04.607647 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:07:04.607660 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:07:04.607767 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.368152099 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-780461d8-1b2d-4b35-8d46-2b00d51ae331" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:07:04.607789 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 32 lines ... I0109 08:08:32.374461 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034","csi.storage.k8s.io/pvc/name":"pvc-9bx47","csi.storage.k8s.io/pvc/namespace":"azuredisk-5466","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034"} I0109 08:08:32.397334 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:08:32.397854 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:08:32.397901 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 to node k8s-agentpool-33023206-vmss000001 I0109 08:08:32.398070 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 false 0})] I0109 08:08:32.398198 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 false 0})]) I0109 08:08:32.607704 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:08:42.727259 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:08:42.727296 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:08:42.727318 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:08:42.727333 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:08:42.727537 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.329531273 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:08:42.727561 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 40 lines ... I0109 08:09:41.653901 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d","csi.storage.k8s.io/pvc/name":"pvc-kkl9m","csi.storage.k8s.io/pvc/namespace":"azuredisk-2790","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d"} I0109 08:09:41.676208 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:09:41.676532 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:09:41.676562 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d to node k8s-agentpool-33023206-vmss000001 I0109 08:09:41.676613 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d false 0})] I0109 08:09:41.676646 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d false 0})]) I0109 08:09:41.856604 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:09:51.963570 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:09:51.963602 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:09:51.963623 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d attached to node k8s-agentpool-33023206-vmss000001. I0109 08:09:51.963637 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:09:51.963676 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.287140894 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:09:51.963711 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 16 lines ... I0109 08:10:29.058761 1 azure_controller_common.go:398] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d from node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d:pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d] E0109 08:10:29.058885 1 azure_controller_vmss.go:202] detach azure disk on node(k8s-agentpool-33023206-vmss000001): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d:pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d]) not found I0109 08:10:29.058942 1 azure_controller_vmss.go:239] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d:pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d]) I0109 08:10:34.027971 1 utils.go:77] GRPC call: /csi.v1.Controller/DeleteVolume I0109 08:10:34.027996 1 utils.go:78] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d"} I0109 08:10:34.028088 1 controllerserver.go:317] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d) I0109 08:10:34.028106 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d) since it's in attaching or detaching state I0109 08:10:34.028163 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=3.19e-05 request="azuredisk_csi_driver_controller_delete_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d" result_code="failed_csi_driver_controller_delete_volume" E0109 08:10:34.028187 1 utils.go:82] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d) since it's in attaching or detaching state I0109 08:10:34.285017 1 azure_controller_vmss.go:252] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d:pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d]) returned with <nil> I0109 08:10:34.285079 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:10:34.285098 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:10:34.285109 1 azure_controller_common.go:422] azureDisk - detach disk(pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d) succeeded I0109 08:10:34.285122 1 controllerserver.go:480] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d from node k8s-agentpool-33023206-vmss000001 successfully I0109 08:10:34.285161 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.226564192 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" ... skipping 20 lines ... I0109 08:11:19.544386 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01","csi.storage.k8s.io/pvc/name":"pvc-ckbm4","csi.storage.k8s.io/pvc/namespace":"azuredisk-5356","requestedsizegib":"10","resourceGroup":"azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01"} I0109 08:11:19.566780 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1238 I0109 08:11:19.567133 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:11:19.567176 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 to node k8s-agentpool-33023206-vmss000001 I0109 08:11:19.567293 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 false 0})] I0109 08:11:19.567336 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 false 0})]) I0109 08:11:19.715020 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:11:29.858527 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:11:29.858573 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:11:29.858602 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:11:29.858621 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:11:29.858677 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.291528833 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:11:29.858705 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 47 lines ... I0109 08:12:43.199901 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216 to node k8s-agentpool-33023206-vmss000001 I0109 08:12:43.199955 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-203bfe9f-490e-4e21-9c2d-677f95626216 false 0})] I0109 08:12:43.200103 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-203bfe9f-490e-4e21-9c2d-677f95626216 false 0})]) I0109 08:12:43.223291 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1238 I0109 08:12:43.223681 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:12:43.223710 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa to node k8s-agentpool-33023206-vmss000001 I0109 08:12:44.130714 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-203bfe9f-490e-4e21-9c2d-677f95626216 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:12:54.266843 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:12:54.266886 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:12:54.266920 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:12:54.266937 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:12:54.266981 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=11.06711047 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-61b92dc0-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-203bfe9f-490e-4e21-9c2d-677f95626216" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:12:54.266999 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0109 08:12:54.267031 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa lun 1 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa false 1})] I0109 08:12:54.267097 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa false 1})]) I0109 08:12:54.484514 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/microsoft.compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:13:04.607638 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:13:04.607670 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:13:04.607692 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa attached to node k8s-agentpool-33023206-vmss000001. I0109 08:13:04.607706 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:13:04.607749 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=21.384059078 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-622c5e4a-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-e9efe891-5be6-4b47-93d6-bee9fa3040aa" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:13:04.607768 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 54 lines ... I0109 08:16:17.545265 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1207 I0109 08:16:17.592493 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 24988 I0109 08:16:17.595009 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:16:17.595050 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb to node k8s-agentpool-33023206-vmss000001 I0109 08:16:17.595087 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb false 0})] I0109 08:16:17.595135 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb false 0})]) I0109 08:16:17.768940 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:16:58.124210 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:16:58.124246 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:16:58.124267 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb attached to node k8s-agentpool-33023206-vmss000001. I0109 08:16:58.124281 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:16:58.124323 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=40.578816852 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-4e9ed40a-ceb1-46f9-a2f1-81b6867e49bb" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:16:58.124343 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 32 lines ... I0109 08:18:11.749916 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-53840185-f8e2-4b53-a1f3-6dc914df13be","csi.storage.k8s.io/pvc/name":"pvc-57tgz","csi.storage.k8s.io/pvc/namespace":"azuredisk-2888","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be"} I0109 08:18:11.776996 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:18:11.777368 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-53840185-f8e2-4b53-a1f3-6dc914df13be. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:18:11.777401 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be to node k8s-agentpool-33023206-vmss000001 I0109 08:18:11.777435 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-53840185-f8e2-4b53-a1f3-6dc914df13be false 0})] I0109 08:18:11.777468 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-53840185-f8e2-4b53-a1f3-6dc914df13be false 0})]) I0109 08:18:11.961745 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-53840185-f8e2-4b53-a1f3-6dc914df13be false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:18:22.067356 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:18:22.067401 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:18:22.067423 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be attached to node k8s-agentpool-33023206-vmss000001. I0109 08:18:22.067589 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:18:22.067692 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.290289812 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-53840185-f8e2-4b53-a1f3-6dc914df13be" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:18:22.067735 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 19 lines ... I0109 08:18:36.051409 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000000","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f91df02f-9a10-41af-8575-d72d658da1e9","csi.storage.k8s.io/pvc/name":"pvc-65sdp","csi.storage.k8s.io/pvc/namespace":"azuredisk-2888","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9"} I0109 08:18:36.077020 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:18:36.077449 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-f91df02f-9a10-41af-8575-d72d658da1e9. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9 to node k8s-agentpool-33023206-vmss000000 (vmState Succeeded). I0109 08:18:36.077479 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9 to node k8s-agentpool-33023206-vmss000000 I0109 08:18:36.077514 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9 lun 0 to node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f91df02f-9a10-41af-8575-d72d658da1e9 false 0})] I0109 08:18:36.077577 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f91df02f-9a10-41af-8575-d72d658da1e9 false 0})]) I0109 08:18:36.428908 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f91df02f-9a10-41af-8575-d72d658da1e9 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:18:46.615225 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000000) successfully I0109 08:18:46.615279 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000000) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:18:46.615301 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9 attached to node k8s-agentpool-33023206-vmss000000. I0109 08:18:46.615316 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9 to node k8s-agentpool-33023206-vmss000000 successfully I0109 08:18:46.615356 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.537913525 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9" node="k8s-agentpool-33023206-vmss000000" result_code="succeeded" I0109 08:18:46.615374 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 11 lines ... I0109 08:19:00.337084 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2","csi.storage.k8s.io/pvc/name":"pvc-wtrm6","csi.storage.k8s.io/pvc/namespace":"azuredisk-2888","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2"} I0109 08:19:00.360676 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:19:00.360923 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:19:00.360956 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 to node k8s-agentpool-33023206-vmss000001 I0109 08:19:00.360999 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 lun 1 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 false 1})] I0109 08:19:00.361049 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 false 1})]) I0109 08:19:00.559634 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:19:10.737383 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:19:10.737421 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:19:10.737441 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:19:10.737457 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:19:10.737655 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.376735202 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-276138ea-9838-4f0d-8f9b-be3e7823d7d2" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:19:10.737678 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 37 lines ... I0109 08:21:09.734248 1 azure_controller_common.go:398] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9 from node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9:pvc-f91df02f-9a10-41af-8575-d72d658da1e9] E0109 08:21:09.734343 1 azure_controller_vmss.go:202] detach azure disk on node(k8s-agentpool-33023206-vmss000000): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9:pvc-f91df02f-9a10-41af-8575-d72d658da1e9]) not found I0109 08:21:09.734360 1 azure_controller_vmss.go:239] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9:pvc-f91df02f-9a10-41af-8575-d72d658da1e9]) I0109 08:21:14.414965 1 utils.go:77] GRPC call: /csi.v1.Controller/DeleteVolume I0109 08:21:14.414993 1 utils.go:78] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9"} I0109 08:21:14.415121 1 controllerserver.go:317] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9) I0109 08:21:14.415147 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9) since it's in attaching or detaching state I0109 08:21:14.415218 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=4.9e-05 request="azuredisk_csi_driver_controller_delete_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9" result_code="failed_csi_driver_controller_delete_volume" E0109 08:21:14.415241 1 utils.go:82] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9) since it's in attaching or detaching state I0109 08:21:14.951642 1 azure_controller_vmss.go:252] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9:pvc-f91df02f-9a10-41af-8575-d72d658da1e9]) returned with <nil> I0109 08:21:14.951701 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000000) successfully I0109 08:21:14.951718 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000000) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:21:14.951730 1 azure_controller_common.go:422] azureDisk - detach disk(pvc-f91df02f-9a10-41af-8575-d72d658da1e9, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9) succeeded I0109 08:21:14.951743 1 controllerserver.go:480] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9 from node k8s-agentpool-33023206-vmss000000 successfully I0109 08:21:14.951781 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.217627069 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f91df02f-9a10-41af-8575-d72d658da1e9" node="k8s-agentpool-33023206-vmss000000" result_code="succeeded" ... skipping 42 lines ... I0109 08:23:02.479116 1 azure_vmss_cache.go:327] refresh the cache of NonVmssUniformNodesCache in rg map[kubetest-y4vzlmmp:{}] I0109 08:23:02.507920 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 12 I0109 08:23:02.508017 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:23:02.508050 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a to node k8s-agentpool-33023206-vmss000001 I0109 08:23:02.508103 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a false 0})] I0109 08:23:02.508145 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a false 0})]) I0109 08:23:02.686804 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:23:12.805839 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:23:12.805878 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:23:12.805899 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a attached to node k8s-agentpool-33023206-vmss000001. I0109 08:23:12.805913 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:23:12.805956 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.326826989 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-01d5d96c-e104-4bd2-bfbd-41306f3afb7a" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:23:12.805975 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 57 lines ... I0109 08:25:52.613115 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d","csi.storage.k8s.io/pvc/name":"pvc-stvct","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d"} I0109 08:25:52.656477 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1217 I0109 08:25:52.656922 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:25:52.656980 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d to node k8s-agentpool-33023206-vmss000001 I0109 08:25:52.657036 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d false 0})] I0109 08:25:52.657094 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d false 0})]) I0109 08:25:52.807224 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:26:02.943259 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:26:02.943294 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:26:02.943314 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d attached to node k8s-agentpool-33023206-vmss000001. I0109 08:26:02.943331 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:26:02.943372 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.286471958 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:26:02.943393 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 25 lines ... I0109 08:26:30.067260 1 azure_controller_common.go:422] azureDisk - detach disk(pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d) succeeded I0109 08:26:30.067283 1 controllerserver.go:480] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d from node k8s-agentpool-33023206-vmss000001 successfully I0109 08:26:30.067383 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=15.259012776 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7a7dad5f-75b1-49b4-97fd-1f91e767914d" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:26:30.067401 1 utils.go:84] GRPC response: {} I0109 08:26:30.067493 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b false 0})] I0109 08:26:30.067526 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b false 0})]) I0109 08:26:30.270581 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:26:40.380216 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:26:40.380249 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:26:40.380272 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b attached to node k8s-agentpool-33023206-vmss000001. I0109 08:26:40.380287 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:26:40.380338 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=18.402593532 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-341a4ac3-0cb8-4225-8ce7-e6bd6281348b" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:26:40.380364 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 40 lines ... I0109 08:27:54.669650 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc","csi.storage.k8s.io/pvc/name":"pvc-fpj2n","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc"} I0109 08:27:54.700083 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1191 I0109 08:27:54.700522 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:27:54.700571 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc to node k8s-agentpool-33023206-vmss000001 I0109 08:27:54.700626 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc false 0})] I0109 08:27:54.700675 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc false 0})]) I0109 08:27:54.857476 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:28:04.993092 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:28:04.993127 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:28:04.993149 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc attached to node k8s-agentpool-33023206-vmss000001. I0109 08:28:04.993163 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:28:04.993203 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.292700823 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:28:04.993222 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 12 lines ... I0109 08:28:23.995759 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-940451f5-01f8-4dce-ad1a-3397422645ef","csi.storage.k8s.io/pvc/name":"pvc-7v8kh","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"20","resizeRequired":"true","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef"} I0109 08:28:24.019426 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1446 I0109 08:28:24.019719 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-940451f5-01f8-4dce-ad1a-3397422645ef. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:28:24.019753 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef to node k8s-agentpool-33023206-vmss000001 I0109 08:28:24.019791 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef lun 1 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-940451f5-01f8-4dce-ad1a-3397422645ef false 1})] I0109 08:28:24.019832 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-940451f5-01f8-4dce-ad1a-3397422645ef false 1})]) I0109 08:28:24.213712 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-940451f5-01f8-4dce-ad1a-3397422645ef false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:28:24.907324 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume I0109 08:28:24.907528 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc"} I0109 08:28:24.907739 1 controllerserver.go:471] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7c94425e-d212-459d-99dc-c5b9bb5ab3cc from node k8s-agentpool-33023206-vmss000001 I0109 08:28:34.396676 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:28:34.396709 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:28:34.396736 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-940451f5-01f8-4dce-ad1a-3397422645ef attached to node k8s-agentpool-33023206-vmss000001. ... skipping 91 lines ... I0109 08:29:35.879112 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:29:35.879366 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:29:35.879393 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae to node k8s-agentpool-33023206-vmss000001 I0109 08:29:35.882374 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:29:35.882607 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-cef873dd-55b6-439b-96a3-97afb644ac60. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-cef873dd-55b6-439b-96a3-97afb644ac60 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:29:35.882635 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-cef873dd-55b6-439b-96a3-97afb644ac60 to node k8s-agentpool-33023206-vmss000001 I0109 08:29:36.589922 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f6a45026-fd85-4354-8b28-e2fd09f0520f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f6a45026-fd85-4354-8b28-e2fd09f0520f false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:29:46.758007 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:29:46.758059 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:29:46.758091 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f6a45026-fd85-4354-8b28-e2fd09f0520f attached to node k8s-agentpool-33023206-vmss000001. I0109 08:29:46.758106 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f6a45026-fd85-4354-8b28-e2fd09f0520f to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:29:46.758149 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.880526656 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f6a45026-fd85-4354-8b28-e2fd09f0520f" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:29:46.758171 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0109 08:29:46.758329 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae lun 1 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-cef873dd-55b6-439b-96a3-97afb644ac60:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cef873dd-55b6-439b-96a3-97afb644ac60 false 2})] I0109 08:29:46.758375 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-cef873dd-55b6-439b-96a3-97afb644ac60:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cef873dd-55b6-439b-96a3-97afb644ac60 false 2})]) I0109 08:29:46.935902 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-cef873dd-55b6-439b-96a3-97afb644ac60:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cef873dd-55b6-439b-96a3-97afb644ac60 false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:29:57.017404 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:29:57.017449 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:29:57.017480 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae attached to node k8s-agentpool-33023206-vmss000001. I0109 08:29:57.017497 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:29:57.017594 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=21.138163168 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-41b33f14-e7ee-437c-8bee-5dcc5fa012ae" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:29:57.017618 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 90 lines ... I0109 08:31:22.050760 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635 to node k8s-agentpool-33023206-vmss000001 I0109 08:31:22.050792 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1c8219f-d465-4e36-b982-f3533557c635 false 0})] I0109 08:31:22.050875 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1c8219f-d465-4e36-b982-f3533557c635 false 0})]) I0109 08:31:22.052289 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:31:22.052574 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:31:22.052600 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 to node k8s-agentpool-33023206-vmss000001 I0109 08:31:22.180011 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a1c8219f-d465-4e36-b982-f3533557c635 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:31:32.324098 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:31:32.324132 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:31:32.324165 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:31:32.324182 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:31:32.324222 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.273485634 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-a1c8219f-d465-4e36-b982-f3533557c635" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:31:32.324249 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0109 08:31:32.324288 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 lun 1 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 false 1})] I0109 08:31:32.324321 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 false 1})]) I0109 08:31:32.448901 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:31:42.532279 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:31:42.532317 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:31:42.532339 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:31:42.532354 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:31:42.532395 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=20.479808512 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:31:42.532493 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 74 lines ... I0109 08:33:22.892993 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654","csi.storage.k8s.io/pvc/name":"pvc-x5z8s","csi.storage.k8s.io/pvc/namespace":"azuredisk-8582","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654"} I0109 08:33:22.939434 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:33:22.939809 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:33:22.939846 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 to node k8s-agentpool-33023206-vmss000001 I0109 08:33:22.939879 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 false 0})] I0109 08:33:22.939981 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 false 0})]) I0109 08:33:23.067055 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:33:33.216735 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:33:33.216793 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:33:33.216830 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:33:33.217135 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:33:33.217284 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.277394646 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:33:33.217316 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 34 lines ... I0109 08:34:10.536107 1 azure_controller_common.go:422] azureDisk - detach disk(pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654) succeeded I0109 08:34:10.536133 1 controllerserver.go:480] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654 from node k8s-agentpool-33023206-vmss000001 successfully I0109 08:34:10.536170 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=15.295678528 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:34:10.536193 1 utils.go:84] GRPC response: {} I0109 08:34:10.536283 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c false 0})] I0109 08:34:10.536319 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c false 0})]) I0109 08:34:10.718101 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:34:20.852219 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:34:20.852251 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:34:20.852271 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c attached to node k8s-agentpool-33023206-vmss000001. I0109 08:34:20.852286 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:34:20.852324 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=17.005779039 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-236d32c1-1fc5-4920-b573-14bdb6896f9c" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:34:20.852347 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 47 lines ... I0109 08:36:53.750390 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:36:53.803647 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 24988 I0109 08:36:53.807018 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-51325031-637b-472d-90e9-6377c66a3f50. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:36:53.807078 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50 to node k8s-agentpool-33023206-vmss000001 I0109 08:36:53.807134 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-51325031-637b-472d-90e9-6377c66a3f50 false 0})] I0109 08:36:53.807194 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-51325031-637b-472d-90e9-6377c66a3f50 false 0})]) I0109 08:36:53.971341 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-51325031-637b-472d-90e9-6377c66a3f50 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:37:04.204062 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:37:04.204118 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:37:04.204153 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:37:04.204388 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:37:04.204474 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.453777143 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-51325031-637b-472d-90e9-6377c66a3f50" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:37:04.204508 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 29 lines ... I0109 08:37:36.052597 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000000","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f","csi.storage.k8s.io/pvc/name":"pvc-cff7n","csi.storage.k8s.io/pvc/namespace":"azuredisk-7726","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f"} I0109 08:37:36.078056 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1501 I0109 08:37:36.078444 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f to node k8s-agentpool-33023206-vmss000000 (vmState Succeeded). I0109 08:37:36.078476 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f to node k8s-agentpool-33023206-vmss000000 I0109 08:37:36.078508 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f lun 0 to node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f false 0})] I0109 08:37:36.078550 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f false 0})]) I0109 08:37:36.243685 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:37:46.372976 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000000) successfully I0109 08:37:46.373014 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000000) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:37:46.373034 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f attached to node k8s-agentpool-33023206-vmss000000. I0109 08:37:46.373049 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f to node k8s-agentpool-33023206-vmss000000 successfully I0109 08:37:46.373093 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.294648259 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d8332487-b4e6-4a03-bcbb-8d3fd54a5f3f" node="k8s-agentpool-33023206-vmss000000" result_code="succeeded" I0109 08:37:46.373112 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 88 lines ... I0109 08:40:10.145604 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-e7b57a9b-cefd-4121-90a3-986676f50f03 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-e7b57a9b-cefd-4121-90a3-986676f50f03:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e7b57a9b-cefd-4121-90a3-986676f50f03 false 0})] I0109 08:40:10.145739 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-e7b57a9b-cefd-4121-90a3-986676f50f03:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e7b57a9b-cefd-4121-90a3-986676f50f03 false 0})]) I0109 08:40:10.146123 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-f524f785-99e1-479d-b5bd-9c283d13182d. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:40:10.146312 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d to node k8s-agentpool-33023206-vmss000001 I0109 08:40:10.146211 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:40:10.146360 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4 to node k8s-agentpool-33023206-vmss000001 I0109 08:40:10.832745 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-e7b57a9b-cefd-4121-90a3-986676f50f03:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-e7b57a9b-cefd-4121-90a3-986676f50f03 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:40:20.917803 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:40:20.917838 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:40:20.917862 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-e7b57a9b-cefd-4121-90a3-986676f50f03 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:40:20.918037 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-e7b57a9b-cefd-4121-90a3-986676f50f03 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:40:20.917936 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d lun 1 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4 false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f524f785-99e1-479d-b5bd-9c283d13182d false 1})] I0109 08:40:20.918244 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4 false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f524f785-99e1-479d-b5bd-9c283d13182d false 1})]) I0109 08:40:20.918643 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.792242556 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-e7b57a9b-cefd-4121-90a3-986676f50f03" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:40:20.918669 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0109 08:40:21.084479 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-baa2668c-2edd-4c15-8f8a-e33ea4e804e4 false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f524f785-99e1-479d-b5bd-9c283d13182d false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:40:36.192735 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:40:36.192772 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:40:36.192800 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d attached to node k8s-agentpool-33023206-vmss000001. I0109 08:40:36.192816 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:40:36.192858 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=26.054951323 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f524f785-99e1-479d-b5bd-9c283d13182d" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:40:36.192908 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 82 lines ... I0109 08:42:21.417970 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-614aeeed-c692-4859-a072-518f1feada12","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-dndmc-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1387","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12"} I0109 08:42:21.444338 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0109 08:42:21.444934 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-614aeeed-c692-4859-a072-518f1feada12. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:42:21.444983 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 to node k8s-agentpool-33023206-vmss000001 I0109 08:42:21.445188 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-614aeeed-c692-4859-a072-518f1feada12 false 0})] I0109 08:42:21.445254 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-614aeeed-c692-4859-a072-518f1feada12 false 0})]) I0109 08:42:21.747588 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-614aeeed-c692-4859-a072-518f1feada12 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:42:36.895182 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:42:36.895235 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:42:36.895557 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:42:36.895667 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:42:36.895804 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=15.450843402 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:42:36.895884 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 47 lines ... I0109 08:45:30.141800 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-614aeeed-c692-4859-a072-518f1feada12","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-dndmc-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1387","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12"} I0109 08:45:30.163225 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0109 08:45:30.163722 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-614aeeed-c692-4859-a072-518f1feada12. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:45:30.163761 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 to node k8s-agentpool-33023206-vmss000001 I0109 08:45:30.163795 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-614aeeed-c692-4859-a072-518f1feada12 false 0})] I0109 08:45:30.163833 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-614aeeed-c692-4859-a072-518f1feada12 false 0})]) I0109 08:45:30.374441 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-614aeeed-c692-4859-a072-518f1feada12 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:45:45.514428 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:45:45.514463 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:45:45.514484 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:45:45.514498 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:45:45.514541 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=15.350817647 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-614aeeed-c692-4859-a072-518f1feada12" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:45:45.514559 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 11 lines ... I0109 08:46:14.511879 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000000","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982","csi.storage.k8s.io/pvc/name":"pvc-wd6tt","csi.storage.k8s.io/pvc/namespace":"azuredisk-4801","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com","tags":"disk=test"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982"} I0109 08:46:14.533037 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1214 I0109 08:46:14.533575 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 to node k8s-agentpool-33023206-vmss000000 (vmState Succeeded). I0109 08:46:14.533615 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 to node k8s-agentpool-33023206-vmss000000 I0109 08:46:14.533696 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 lun 0 to node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 false 0})] I0109 08:46:14.533776 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 false 0})]) I0109 08:46:14.692180 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:46:29.801189 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000000) successfully I0109 08:46:29.801223 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000000) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:46:29.801242 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 attached to node k8s-agentpool-33023206-vmss000000. I0109 08:46:29.801255 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982 to node k8s-agentpool-33023206-vmss000000 successfully I0109 08:46:29.801294 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=15.267726426 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-6cf2c553-c6a1-4d58-9ee8-cabb486e3982" node="k8s-agentpool-33023206-vmss000000" result_code="succeeded" I0109 08:46:29.801313 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 54 lines ... I0109 08:47:46.515703 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0","csi.storage.k8s.io/pvc/name":"pvc-gkf9d","csi.storage.k8s.io/pvc/namespace":"azuredisk-8154","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0"} I0109 08:47:46.536558 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0109 08:47:46.537018 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:47:46.537047 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 to node k8s-agentpool-33023206-vmss000001 I0109 08:47:46.537101 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 false 0})] I0109 08:47:46.537196 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 false 0})]) I0109 08:47:46.687509 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:47:56.808661 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:47:56.808697 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:47:56.808716 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:47:56.808731 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:47:56.808772 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.271754141 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-ac80957c-bdc8-41f3-be3a-105cd7b17de0" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:47:56.808792 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 31 lines ... I0109 08:49:23.673795 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-2hllx-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1166","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426"} I0109 08:49:23.713871 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0109 08:49:23.714337 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:49:23.714375 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 to node k8s-agentpool-33023206-vmss000001 I0109 08:49:23.714570 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 false 0})] I0109 08:49:23.714641 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 false 0})]) I0109 08:49:23.880612 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:49:34.163513 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:49:34.163571 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:49:34.163604 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:49:34.163629 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:49:34.163941 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.449339821 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:49:34.163996 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 35 lines ... I0109 08:50:46.654171 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-2hllx-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1166","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426"} I0109 08:50:46.699173 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0109 08:50:46.699594 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:50:46.699623 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 to node k8s-agentpool-33023206-vmss000001 I0109 08:50:46.699763 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 false 0})] I0109 08:50:46.699810 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 false 0})]) I0109 08:50:46.833605 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:50:56.910780 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:50:56.910814 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:50:56.910835 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:50:56.910851 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:50:56.910906 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.211293237 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-33c4910b-d15b-4a21-9fb5-1125a65a6426" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:50:56.910928 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 19 lines ... I0109 08:51:11.429324 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000000","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f46f26f7-2b90-4582-866b-720a5a416a1a","csi.storage.k8s.io/pvc/name":"pvc-d7l6f","csi.storage.k8s.io/pvc/namespace":"azuredisk-783","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a"} I0109 08:51:11.460862 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1192 I0109 08:51:11.461343 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-f46f26f7-2b90-4582-866b-720a5a416a1a. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a to node k8s-agentpool-33023206-vmss000000 (vmState Succeeded). I0109 08:51:11.461390 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a to node k8s-agentpool-33023206-vmss000000 I0109 08:51:11.461423 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a lun 0 to node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f46f26f7-2b90-4582-866b-720a5a416a1a false 0})] I0109 08:51:11.461460 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f46f26f7-2b90-4582-866b-720a5a416a1a false 0})]) I0109 08:51:11.687942 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f46f26f7-2b90-4582-866b-720a5a416a1a false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:51:21.768301 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000000) successfully I0109 08:51:21.768336 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000000) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:51:21.768358 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a attached to node k8s-agentpool-33023206-vmss000000. I0109 08:51:21.768372 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a to node k8s-agentpool-33023206-vmss000000 successfully I0109 08:51:21.768413 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.307070893 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a" node="k8s-agentpool-33023206-vmss000000" result_code="succeeded" I0109 08:51:21.768432 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 35 lines ... I0109 08:52:21.824925 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f46f26f7-2b90-4582-866b-720a5a416a1a","csi.storage.k8s.io/pvc/name":"pvc-d7l6f","csi.storage.k8s.io/pvc/namespace":"azuredisk-783","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a"} I0109 08:52:21.849988 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1192 I0109 08:52:21.850257 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-f46f26f7-2b90-4582-866b-720a5a416a1a. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:52:21.850302 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a to node k8s-agentpool-33023206-vmss000001 I0109 08:52:21.850337 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f46f26f7-2b90-4582-866b-720a5a416a1a false 0})] I0109 08:52:21.850369 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f46f26f7-2b90-4582-866b-720a5a416a1a false 0})]) I0109 08:52:22.017733 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f46f26f7-2b90-4582-866b-720a5a416a1a false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:52:32.096100 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:52:32.096156 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:52:32.096191 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a attached to node k8s-agentpool-33023206-vmss000001. I0109 08:52:32.096218 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:52:32.096283 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.246009257 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-f46f26f7-2b90-4582-866b-720a5a416a1a" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:52:32.096307 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 34 lines ... I0109 08:53:57.534748 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:53:57.534786 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 to node k8s-agentpool-33023206-vmss000001 I0109 08:53:57.534822 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6:%!s(*provider.AttachDiskOptions=&{None pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 false 0})] I0109 08:53:57.534861 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6:%!s(*provider.AttachDiskOptions=&{None pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 false 0})]) I0109 08:53:57.701435 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerPublishVolume I0109 08:53:57.701470 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000000","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":5}},"volume_context":{"cachingmode":"None","csi.storage.k8s.io/pv/name":"pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6","csi.storage.k8s.io/pvc/name":"pvc-lrvvh","csi.storage.k8s.io/pvc/namespace":"azuredisk-7920","maxshares":"2","requestedsizegib":"10","skuname":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6"} I0109 08:53:57.731184 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6:%!s(*provider.AttachDiskOptions=&{None pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:53:57.760626 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1214 I0109 08:53:57.760953 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 to node k8s-agentpool-33023206-vmss000000 (vmState Succeeded). I0109 08:53:57.760988 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 to node k8s-agentpool-33023206-vmss000000 I0109 08:53:57.761022 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 lun 0 to node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6:%!s(*provider.AttachDiskOptions=&{None pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 false 0})] I0109 08:53:57.761058 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6:%!s(*provider.AttachDiskOptions=&{None pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 false 0})]) I0109 08:53:58.831797 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6:%!s(*provider.AttachDiskOptions=&{None pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:54:17.902440 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:54:17.902472 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:54:17.902492 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:54:17.902504 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:54:17.902562 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=20.367800645 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:54:17.902584 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 60 lines ... I0109 08:55:46.493310 1 azure_vmss_cache.go:327] refresh the cache of NonVmssUniformNodesCache in rg map[kubetest-y4vzlmmp:{}] I0109 08:55:46.513914 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 12 I0109 08:55:46.514004 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:55:46.514038 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e to node k8s-agentpool-33023206-vmss000001 I0109 08:55:46.514076 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e false 0})] I0109 08:55:46.514112 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e false 0})]) I0109 08:55:46.684323 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:55:56.848675 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:55:56.848783 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:55:56.848832 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e attached to node k8s-agentpool-33023206-vmss000001. I0109 08:55:56.848861 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:55:56.849190 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.355710727 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:55:56.849235 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 44 lines ... I0109 08:56:53.885423 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3e7bff55-8880-4a95-8e00-f95a8695258a","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a"} I0109 08:56:53.905651 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1219 I0109 08:56:53.906008 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-3e7bff55-8880-4a95-8e00-f95a8695258a. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:56:53.906072 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a to node k8s-agentpool-33023206-vmss000001 I0109 08:56:53.906128 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a lun 0 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3e7bff55-8880-4a95-8e00-f95a8695258a false 0})] I0109 08:56:53.906213 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3e7bff55-8880-4a95-8e00-f95a8695258a false 0})]) I0109 08:56:54.047512 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3e7bff55-8880-4a95-8e00-f95a8695258a false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:57:04.126506 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:57:04.126540 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:57:04.126625 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a attached to node k8s-agentpool-33023206-vmss000001. I0109 08:57:04.126683 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:57:04.126778 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.220741819 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-3e7bff55-8880-4a95-8e00-f95a8695258a" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:57:04.126863 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 11 lines ... I0109 08:57:18.271065 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0109 08:57:18.323559 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 24988 I0109 08:57:18.326902 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-9361a51e-a2f0-4004-836d-71cb950fe845. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845 to node k8s-agentpool-33023206-vmss000000 (vmState Succeeded). I0109 08:57:18.326942 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845 to node k8s-agentpool-33023206-vmss000000 I0109 08:57:18.326989 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845 lun 0 to node k8s-agentpool-33023206-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9361a51e-a2f0-4004-836d-71cb950fe845 false 0})] I0109 08:57:18.327035 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9361a51e-a2f0-4004-836d-71cb950fe845 false 0})]) I0109 08:57:18.453212 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9361a51e-a2f0-4004-836d-71cb950fe845 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:57:28.539493 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000000) successfully I0109 08:57:28.539532 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000000) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:57:28.539552 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845 attached to node k8s-agentpool-33023206-vmss000000. I0109 08:57:28.539605 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845 to node k8s-agentpool-33023206-vmss000000 successfully I0109 08:57:28.539645 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.268287552 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845" node="k8s-agentpool-33023206-vmss000000" result_code="succeeded" I0109 08:57:28.539667 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 18 lines ... I0109 08:57:44.572282 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-33023206-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8"} I0109 08:57:44.739976 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1256 I0109 08:57:44.740636 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 to node k8s-agentpool-33023206-vmss000001 (vmState Succeeded). I0109 08:57:44.740686 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 to node k8s-agentpool-33023206-vmss000001 I0109 08:57:44.740886 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 lun 1 to node k8s-agentpool-33023206-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 false 1})] I0109 08:57:44.741102 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 false 1})]) I0109 08:57:45.104817 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-y4vzlmmp): vm(k8s-agentpool-33023206-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-y4vzlmmp/providers/microsoft.compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0109 08:58:00.248866 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss, k8s-agentpool-33023206-vmss000001) successfully I0109 08:58:00.248905 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-33023206-vmss, kubetest-y4vzlmmp, k8s-agentpool-33023206-vmss000001) for cacheKey(kubetest-y4vzlmmp/k8s-agentpool-33023206-vmss) updated successfully I0109 08:58:00.248954 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 attached to node k8s-agentpool-33023206-vmss000001. I0109 08:58:00.248974 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8 to node k8s-agentpool-33023206-vmss000001 successfully I0109 08:58:00.249097 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=15.508400529 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-y4vzlmmp" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-d17506f8-afdf-4cd7-8ca6-cc9bc0fc3fc8" node="k8s-agentpool-33023206-vmss000001" result_code="succeeded" I0109 08:58:00.249187 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 20 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0109 08:04:21.908648 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 e2e-test I0109 08:04:21.909199 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0109 08:04:21.935089 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0109 08:04:21.935112 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0109 08:04:21.935121 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0109 08:04:21.935177 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0109 08:04:21.935971 1 azure_auth.go:245] Using AzurePublicCloud environment I0109 08:04:21.936021 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I0109 08:04:21.936065 1 azure.go:777] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 25 lines ... I0109 08:04:21.936480 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0109 08:04:21.936513 1 azure_vmasclient.go:70] Azure AvailabilitySetsClient (read ops) using rate limit config: QPS=6, bucket=20 I0109 08:04:21.936524 1 azure_vmasclient.go:73] Azure AvailabilitySetsClient (write ops) using rate limit config: QPS=100, bucket=1000 I0109 08:04:21.936591 1 azure.go:1008] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0109 08:04:21.936616 1 azuredisk.go:192] disable UseInstanceMetadata for controller I0109 08:04:21.936625 1 azuredisk.go:204] cloud: AzurePublicCloud, location: westus2, rg: kubetest-y4vzlmmp, VMType: vmss, PrimaryScaleSetName: k8s-agentpool-33023206-vmss, PrimaryAvailabilitySetName: , DisableAvailabilitySetNodes: false I0109 08:04:21.940785 1 mount_linux.go:287] 'umount /tmp/kubelet-detect-safe-umount6807372' failed with: exit status 32, output: umount: /tmp/kubelet-detect-safe-umount6807372: must be superuser to unmount. I0109 08:04:21.940836 1 mount_linux.go:289] Detected umount with unsafe 'not mounted' behavior I0109 08:04:21.940922 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0109 08:04:21.940950 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0109 08:04:21.940958 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0109 08:04:21.940965 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0109 08:04:21.940987 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 62 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0109 08:04:17.469734 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 e2e-test I0109 08:04:17.470390 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0109 08:04:17.499838 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0109 08:04:17.499895 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0109 08:04:17.499905 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0109 08:04:17.499933 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0109 08:04:17.500758 1 azure_auth.go:245] Using AzurePublicCloud environment I0109 08:04:17.500797 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I0109 08:04:17.500823 1 azure.go:777] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 148 lines ... I0109 08:08:50.057445 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0109 08:08:50.070424 1 mount_linux.go:570] Output: "" I0109 08:08:50.070476 1 mount_linux.go:529] Disk "/dev/disk/azure/scsi1/lun0" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/scsi1/lun0] I0109 08:08:50.551288 1 mount_linux.go:539] Disk successfully formatted (mkfs): ext4 - /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount I0109 08:08:50.551320 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount I0109 08:08:50.551347 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount) E0109 08:08:50.572520 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. E0109 08:08:50.573013 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. I0109 08:08:51.115637 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0109 08:08:51.115672 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034","csi.storage.k8s.io/pvc/name":"pvc-9bx47","csi.storage.k8s.io/pvc/namespace":"azuredisk-5466","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034"} I0109 08:08:52.911997 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0109 08:08:52.912051 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0109 08:08:52.912486 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount with mount options([invalid mount options]) I0109 08:08:52.912515 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0109 08:08:52.921951 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0109 08:08:52.921972 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0109 08:08:52.938926 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount I0109 08:08:52.939021 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount) E0109 08:08:52.958440 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. E0109 08:08:52.958487 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. I0109 08:08:53.976410 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0109 08:08:53.976445 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034","csi.storage.k8s.io/pvc/name":"pvc-9bx47","csi.storage.k8s.io/pvc/namespace":"azuredisk-5466","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034"} I0109 08:08:55.815059 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0109 08:08:55.815110 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0109 08:08:55.815926 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount with mount options([invalid mount options]) I0109 08:08:55.815960 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0109 08:08:55.830753 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0109 08:08:55.830787 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0109 08:08:55.848318 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount I0109 08:08:55.848365 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount) E0109 08:08:55.870606 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. E0109 08:08:55.870656 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-93a6c706-2f25-4d26-ae8b-2feebb0b3034/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. I0109 08:09:57.492357 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0109 08:09:57.492386 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d","csi.storage.k8s.io/pvc/name":"pvc-kkl9m","csi.storage.k8s.io/pvc/namespace":"azuredisk-2790","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d"} I0109 08:09:59.293219 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0109 08:09:59.293284 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0109 08:09:59.293314 1 utils.go:84] GRPC response: {} I0109 08:09:59.307458 1 utils.go:77] GRPC call: /csi.v1.Node/NodePublishVolume ... skipping 16 lines ... I0109 08:10:04.846206 1 utils.go:84] GRPC response: {} I0109 08:10:04.881300 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0109 08:10:04.881324 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d"} I0109 08:10:04.881405 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d I0109 08:10:04.881426 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0109 08:10:04.881440 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d I0109 08:10:04.883412 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d I0109 08:10:04.883434 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d" I0109 08:10:04.883523 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-b83055ef-d7b4-4142-b500-a5a5dd1b171d successfully I0109 08:10:04.883536 1 utils.go:84] GRPC response: {} I0109 08:11:35.312960 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0109 08:11:35.312995 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01","csi.storage.k8s.io/pvc/name":"pvc-ckbm4","csi.storage.k8s.io/pvc/namespace":"azuredisk-5356","requestedsizegib":"10","resourceGroup":"azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-2fc84c51-8ff5-11ed-b8dd-5e7dfbe78601/providers/Microsoft.Compute/disks/pvc-40f2fc78-a1ff-4856-a4dc-1f73b7359b01"} I0109 08:11:37.178094 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 648 lines ... I0109 08:31:59.703588 1 utils.go:84] GRPC response: {} I0109 08:31:59.801581 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0109 08:31:59.801607 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4"} I0109 08:31:59.802214 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 I0109 08:31:59.802375 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0109 08:31:59.802403 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 I0109 08:31:59.804532 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 I0109 08:31:59.804552 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4" I0109 08:31:59.804752 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-c5bed0e8-98d7-4dae-bdf9-43bf487dd1d4 successfully I0109 08:31:59.804834 1 utils.go:84] GRPC response: {} I0109 08:33:38.589837 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0109 08:33:38.589862 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654","csi.storage.k8s.io/pvc/name":"pvc-x5z8s","csi.storage.k8s.io/pvc/namespace":"azuredisk-8582","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-7e2c1289-22e4-4f9a-a663-2bdea653e654"} I0109 08:33:40.382968 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 537 lines ... I0109 08:55:07.276406 1 utils.go:84] GRPC response: {} I0109 08:55:07.309443 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0109 08:55:07.309479 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6"} I0109 08:55:07.309541 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 I0109 08:55:07.309577 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0109 08:55:07.309588 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 I0109 08:55:07.312054 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 I0109 08:55:07.312073 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6" I0109 08:55:07.312157 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 successfully I0109 08:55:07.312169 1 utils.go:84] GRPC response: {} I0109 08:56:02.278736 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0109 08:56:02.278778 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e","csi.storage.k8s.io/pvc/name":"pvc-vp8ht","csi.storage.k8s.io/pvc/namespace":"azuredisk-1092","device-setting/device/queue_depth":"17","device-setting/queue/max_sectors_kb":"211","device-setting/queue/nr_requests":"44","device-setting/queue/read_ahead_kb":"256","device-setting/queue/rotational":"0","device-setting/queue/scheduler":"none","device-setting/queue/wbt_lat_usec":"0","perfProfile":"advanced","requestedsizegib":"10","skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-91206da9-c29f-4194-b3d8-7ac9fb89166e"} I0109 08:56:04.074628 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 100 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0109 08:04:11.898745 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 e2e-test I0109 08:04:11.899278 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0109 08:04:11.936574 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0109 08:04:11.936845 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0109 08:04:11.937061 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0109 08:04:11.937276 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0109 08:04:11.938548 1 azure_auth.go:245] Using AzurePublicCloud environment I0109 08:04:11.938739 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I0109 08:04:11.938943 1 azure.go:777] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 69 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0109 08:04:15.656986 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-22e27a8ea5922e24a5393b23757f258026dbfee7 e2e-test I0109 08:04:15.657492 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0109 08:04:15.687105 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0109 08:04:15.687149 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0109 08:04:15.687160 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0109 08:04:15.687185 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0109 08:04:15.688421 1 azure_auth.go:245] Using AzurePublicCloud environment I0109 08:04:15.688485 1 azure_auth.go:130] azure: using client_id+client_secret to retrieve access token I0109 08:04:15.688539 1 azure.go:777] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 285 lines ... I0109 08:55:07.568598 1 utils.go:84] GRPC response: {} I0109 08:55:07.618874 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0109 08:55:07.618899 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6"} I0109 08:55:07.618950 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 I0109 08:55:07.618975 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0109 08:55:07.618991 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 I0109 08:55:07.621833 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 I0109 08:55:07.621854 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6" I0109 08:55:07.621926 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-5a4bd29d-351f-47c5-a584-e3b420fbe2b6 successfully I0109 08:55:07.621941 1 utils.go:84] GRPC response: {} I0109 08:57:33.981539 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0109 08:57:33.981559 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-9361a51e-a2f0-4004-836d-71cb950fe845/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9361a51e-a2f0-4004-836d-71cb950fe845","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1673251461142-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-y4vzlmmp/providers/Microsoft.Compute/disks/pvc-9361a51e-a2f0-4004-836d-71cb950fe845"} I0109 08:57:35.591076 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 594 lines ... cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-y4vzlmmp",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="300"} 45 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-y4vzlmmp",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="600"} 45 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-y4vzlmmp",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="1200"} 45 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-y4vzlmmp",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="+Inf"} 45 cloudprovider_azure_op_duration_seconds_sum{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-y4vzlmmp",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 844.986671969 cloudprovider_azure_op_duration_seconds_count{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-y4vzlmmp",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 45 # HELP cloudprovider_azure_op_failure_count [ALPHA] Number of failed Azure service operations # TYPE cloudprovider_azure_op_failure_count counter cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_delete_volume",resource_group="kubetest-y4vzlmmp",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 2 # HELP disabled_metric_total [ALPHA] The count of disabled metrics. # TYPE disabled_metric_total counter disabled_metric_total 0 # HELP go_cgo_go_to_c_calls_calls_total Count of calls made from Go to C by the current process. ... skipping 67 lines ... # HELP go_gc_heap_objects_objects Number of objects, live or unswept, occupying heap memory. # TYPE go_gc_heap_objects_objects gauge go_gc_heap_objects_objects 50711 # HELP go_gc_heap_tiny_allocs_objects_total Count of small allocations that are packed together into blocks. These allocations are counted separately from other allocations because each individual allocation is not tracked by the runtime, only their block. Each block is already accounted for in allocs-by-size and frees-by-size. # TYPE go_gc_heap_tiny_allocs_objects_total counter go_gc_heap_tiny_allocs_objects_total 47143 # HELP go_gc_limiter_last_enabled_gc_cycle GC cycle the last time the GC CPU limiter was enabled. This metric is useful for diagnosing the root cause of an out-of-memory error, because the limiter trades memory for CPU time when the GC's CPU time gets too high. This is most likely to occur with use of SetMemoryLimit. The first GC cycle is cycle 1, so a value of 0 indicates that it was never enabled. # TYPE go_gc_limiter_last_enabled_gc_cycle gauge go_gc_limiter_last_enabled_gc_cycle 0 # HELP go_gc_pauses_seconds Distribution individual GC-related stop-the-world pause latencies. # TYPE go_gc_pauses_seconds histogram go_gc_pauses_seconds_bucket{le="9.999999999999999e-10"} 0 go_gc_pauses_seconds_bucket{le="9.999999999999999e-09"} 0 ... skipping 259 lines ... # HELP go_gc_heap_objects_objects Number of objects, live or unswept, occupying heap memory. # TYPE go_gc_heap_objects_objects gauge go_gc_heap_objects_objects 36194 # HELP go_gc_heap_tiny_allocs_objects_total Count of small allocations that are packed together into blocks. These allocations are counted separately from other allocations because each individual allocation is not tracked by the runtime, only their block. Each block is already accounted for in allocs-by-size and frees-by-size. # TYPE go_gc_heap_tiny_allocs_objects_total counter go_gc_heap_tiny_allocs_objects_total 4738 # HELP go_gc_limiter_last_enabled_gc_cycle GC cycle the last time the GC CPU limiter was enabled. This metric is useful for diagnosing the root cause of an out-of-memory error, because the limiter trades memory for CPU time when the GC's CPU time gets too high. This is most likely to occur with use of SetMemoryLimit. The first GC cycle is cycle 1, so a value of 0 indicates that it was never enabled. # TYPE go_gc_limiter_last_enabled_gc_cycle gauge go_gc_limiter_last_enabled_gc_cycle 0 # HELP go_gc_pauses_seconds Distribution individual GC-related stop-the-world pause latencies. # TYPE go_gc_pauses_seconds histogram go_gc_pauses_seconds_bucket{le="9.999999999999999e-10"} 0 go_gc_pauses_seconds_bucket{le="9.999999999999999e-09"} 0 ... skipping 272 lines ... [AfterSuite] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/suite_test.go:165[0m [38;5;243m------------------------------[0m [38;5;9m[1mSummarizing 1 Failure:[0m [38;5;9m[FAIL][0m [0mDynamic Provisioning [38;5;243m[multi-az] [0m[38;5;9m[1m[It] should create a pod, write to its pv, take a volume snapshot, overwrite data in original pv, create another pod from the snapshot, and read unaltered original data from original pv[disk.csi.azure.com][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [38;5;9m[1mRan 26 of 66 Specs in 4306.672 seconds[0m [38;5;9m[1mFAIL![0m -- [38;5;10m[1m25 Passed[0m | [38;5;9m[1m1 Failed[0m | [38;5;11m[1m0 Pending[0m | [38;5;14m[1m40 Skipped[0m [38;5;228mYou're using deprecated Ginkgo functionality:[0m [38;5;228m=============================================[0m [38;5;11mSupport for custom reporters has been removed in V2. Please read the documentation linked to below for Ginkgo's new behavior and for a migration path:[0m [1mLearn more at:[0m [38;5;14m[4mhttps://onsi.github.io/ginkgo/MIGRATING_TO_V2#removed-custom-reporters[0m [38;5;243mTo silence deprecations that can be silenced set the following environment variable:[0m [38;5;243mACK_GINKGO_DEPRECATIONS=2.4.0[0m --- FAIL: TestE2E (4306.67s) FAIL FAIL sigs.k8s.io/azuredisk-csi-driver/test/e2e 4306.750s FAIL make: *** [Makefile:261: e2e-test] Error 1 2023/01/09 08:59:12 process.go:155: Step 'make e2e-test' finished in 1h14m52.437539661s 2023/01/09 08:59:12 aksengine_helpers.go:425: downloading /root/tmp2655200364/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh 2023/01/09 08:59:12 util.go:70: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh 2023/01/09 08:59:12 process.go:153: Running: chmod +x /root/tmp2655200364/log-dump.sh 2023/01/09 08:59:12 process.go:155: Step 'chmod +x /root/tmp2655200364/log-dump.sh' finished in 3.96393ms 2023/01/09 08:59:12 aksengine_helpers.go:425: downloading /root/tmp2655200364/log-dump-daemonset.yaml from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump-daemonset.yaml ... skipping 63 lines ... ssh key file /root/.ssh/id_rsa does not exist. Exiting. 2023/01/09 08:59:47 process.go:155: Step 'bash -c /root/tmp2655200364/win-ci-logs-collector.sh kubetest-y4vzlmmp.westus2.cloudapp.azure.com /root/tmp2655200364 /root/.ssh/id_rsa' finished in 3.310737ms 2023/01/09 08:59:47 aksengine.go:1141: Deleting resource group: kubetest-y4vzlmmp. 2023/01/09 09:06:04 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml. 2023/01/09 09:06:04 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}" 2023/01/09 09:06:05 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 1.511582421s 2023/01/09 09:06:05 main.go:328: Something went wrong: encountered 1 errors: [error during make e2e-test: exit status 2] + EXIT_VALUE=1 + set +o xtrace Cleaning up after docker in docker. ================================================================================ Cleaning up after docker b1ccbbe462df ... skipping 4 lines ...