kubernetes-sigs/azuredisk-csi-driver PR #1704 pull-azuredisk-csi-driver-e2e-multi-az #1619333791539531776
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| PR | andyzhangx: fix: switch base image to fix CVEs |
| Result | FAILURE |
| Tests | 1 failed / 13 succeeded |
| Started | |
| Elapsed | 1h35m |
| Revision | a093a52191d3e3d9e4045b29bef24ef84b1ddc4f |
| Refs |
1704 |
| job-version | v1.27.0-alpha.1.69+d7cb1c54a540c9 |
| kubetest-version | v20230117-50d6df3625 |
| revision | v1.27.0-alpha.1.69+d7cb1c54a540c9 |
error during make e2e-test: exit status 2
from junit_runner.xml
Filter through log files | View test history on testgrid
kubetest Check APIReachability
kubetest Deferred TearDown
kubetest DumpClusterLogs
kubetest GetDeployer
kubetest IsUp
kubetest Prepare
kubetest TearDown
kubetest TearDown Previous
kubetest Timeout
kubetest Up
kubetest kubectl version
kubetest list nodes
kubetest test setup
... skipping 518 lines ... type: string type: object oneOf: - required: ["persistentVolumeClaimName"] - required: ["volumeSnapshotContentName"] volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 60 lines ... type: string volumeSnapshotContentName: description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable. type: string type: object volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 254 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 108 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 865 lines ... image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.6.0" args: - "-csi-address=$(ADDRESS)" - "-v=2" - "-leader-election" - "--leader-election-namespace=kube-system" - '-handle-volume-inuse-error=false' - '-feature-gates=RecoverVolumeExpansionFailure=true' - "-timeout=240s" env: - name: ADDRESS value: /csi/csi.sock volumeMounts: ... skipping 216 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:05:38.374[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:05:38.374[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:05:38.427[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:05:38.427[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:05:38.484[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:05:38.484[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:05:38.537[0m Jan 28 14:05:38.537: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-xxzbb" in namespace "azuredisk-8081" to be "Succeeded or Failed" Jan 28 14:05:38.589: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 51.322718ms Jan 28 14:05:40.641: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.104099823s Jan 28 14:05:42.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.104427893s Jan 28 14:05:44.639: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.102070719s Jan 28 14:05:46.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.105100933s Jan 28 14:05:48.641: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.10316767s ... skipping 6 lines ... Jan 28 14:06:02.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 24.104397673s Jan 28 14:06:04.641: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 26.103421192s Jan 28 14:06:06.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 28.10484874s Jan 28 14:06:08.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 30.104653162s Jan 28 14:06:10.647: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Succeeded", Reason="", readiness=false. Elapsed: 32.109480558s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:06:10.647[0m Jan 28 14:06:10.647: INFO: Pod "azuredisk-volume-tester-xxzbb" satisfied condition "Succeeded or Failed" Jan 28 14:06:10.647: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-xxzbb" Jan 28 14:06:10.742: INFO: Pod azuredisk-volume-tester-xxzbb has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-xxzbb in namespace azuredisk-8081 [38;5;243m01/28/23 14:06:10.742[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:06:10.856[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:06:10.906[0m ... skipping 44 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:05:38.374[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:05:38.374[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:05:38.427[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:05:38.427[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:05:38.484[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:05:38.484[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:05:38.537[0m Jan 28 14:05:38.537: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-xxzbb" in namespace "azuredisk-8081" to be "Succeeded or Failed" Jan 28 14:05:38.589: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 51.322718ms Jan 28 14:05:40.641: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.104099823s Jan 28 14:05:42.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.104427893s Jan 28 14:05:44.639: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.102070719s Jan 28 14:05:46.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.105100933s Jan 28 14:05:48.641: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.10316767s ... skipping 6 lines ... Jan 28 14:06:02.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 24.104397673s Jan 28 14:06:04.641: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 26.103421192s Jan 28 14:06:06.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 28.10484874s Jan 28 14:06:08.642: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Pending", Reason="", readiness=false. Elapsed: 30.104653162s Jan 28 14:06:10.647: INFO: Pod "azuredisk-volume-tester-xxzbb": Phase="Succeeded", Reason="", readiness=false. Elapsed: 32.109480558s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:06:10.647[0m Jan 28 14:06:10.647: INFO: Pod "azuredisk-volume-tester-xxzbb" satisfied condition "Succeeded or Failed" Jan 28 14:06:10.647: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-xxzbb" Jan 28 14:06:10.742: INFO: Pod azuredisk-volume-tester-xxzbb has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-xxzbb in namespace azuredisk-8081 [38;5;243m01/28/23 14:06:10.742[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:06:10.856[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:06:10.906[0m ... skipping 39 lines ... Jan 28 14:06:54.661: INFO: PersistentVolumeClaim pvc-m4bmj found but phase is Pending instead of Bound. Jan 28 14:06:56.714: INFO: PersistentVolumeClaim pvc-m4bmj found and phase=Bound (4.156930016s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 14:06:56.714[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:06:56.768[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:06:56.819[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:06:56.819[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m01/28/23 14:06:56.875[0m Jan 28 14:06:56.875: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7m7g4" in namespace "azuredisk-2540" to be "Succeeded or Failed" Jan 28 14:06:56.926: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 50.517278ms Jan 28 14:06:58.980: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.105059341s Jan 28 14:07:00.977: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.101791795s Jan 28 14:07:02.978: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.102861229s Jan 28 14:07:04.978: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.102607787s Jan 28 14:07:06.979: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.103423246s Jan 28 14:07:08.983: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 12.108281683s Jan 28 14:07:10.982: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 14.107360655s Jan 28 14:07:12.982: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 16.106621695s Jan 28 14:07:14.980: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 18.105136033s Jan 28 14:07:16.980: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 20.10504463s Jan 28 14:07:18.981: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 22.105540339s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:07:18.981[0m Jan 28 14:07:18.981: INFO: Pod "azuredisk-volume-tester-7m7g4" satisfied condition "Succeeded or Failed" Jan 28 14:07:18.981: INFO: deleting Pod "azuredisk-2540"/"azuredisk-volume-tester-7m7g4" Jan 28 14:07:19.042: INFO: Pod azuredisk-volume-tester-7m7g4 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7m7g4 in namespace azuredisk-2540 [38;5;243m01/28/23 14:07:19.042[0m Jan 28 14:07:19.108: INFO: deleting PVC "azuredisk-2540"/"pvc-m4bmj" Jan 28 14:07:19.108: INFO: Deleting PersistentVolumeClaim "pvc-m4bmj" ... skipping 38 lines ... Jan 28 14:06:54.661: INFO: PersistentVolumeClaim pvc-m4bmj found but phase is Pending instead of Bound. Jan 28 14:06:56.714: INFO: PersistentVolumeClaim pvc-m4bmj found and phase=Bound (4.156930016s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 14:06:56.714[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:06:56.768[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:06:56.819[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:06:56.819[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m01/28/23 14:06:56.875[0m Jan 28 14:06:56.875: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7m7g4" in namespace "azuredisk-2540" to be "Succeeded or Failed" Jan 28 14:06:56.926: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 50.517278ms Jan 28 14:06:58.980: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.105059341s Jan 28 14:07:00.977: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.101791795s Jan 28 14:07:02.978: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.102861229s Jan 28 14:07:04.978: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.102607787s Jan 28 14:07:06.979: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.103423246s Jan 28 14:07:08.983: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 12.108281683s Jan 28 14:07:10.982: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 14.107360655s Jan 28 14:07:12.982: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 16.106621695s Jan 28 14:07:14.980: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 18.105136033s Jan 28 14:07:16.980: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Pending", Reason="", readiness=false. Elapsed: 20.10504463s Jan 28 14:07:18.981: INFO: Pod "azuredisk-volume-tester-7m7g4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 22.105540339s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:07:18.981[0m Jan 28 14:07:18.981: INFO: Pod "azuredisk-volume-tester-7m7g4" satisfied condition "Succeeded or Failed" Jan 28 14:07:18.981: INFO: deleting Pod "azuredisk-2540"/"azuredisk-volume-tester-7m7g4" Jan 28 14:07:19.042: INFO: Pod azuredisk-volume-tester-7m7g4 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7m7g4 in namespace azuredisk-2540 [38;5;243m01/28/23 14:07:19.042[0m Jan 28 14:07:19.108: INFO: deleting PVC "azuredisk-2540"/"pvc-m4bmj" Jan 28 14:07:19.108: INFO: Deleting PersistentVolumeClaim "pvc-m4bmj" ... skipping 30 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:08:00.762[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:08:00.762[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:08:00.819[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:08:00.819[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:08:00.877[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:08:00.878[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:08:00.954[0m Jan 28 14:08:00.954: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-2xcnl" in namespace "azuredisk-4728" to be "Succeeded or Failed" Jan 28 14:08:01.009: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 54.527747ms Jan 28 14:08:03.064: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110043283s Jan 28 14:08:05.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110744221s Jan 28 14:08:07.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.111270269s Jan 28 14:08:09.066: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111909997s Jan 28 14:08:11.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110373915s ... skipping 16 lines ... Jan 28 14:08:45.066: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 44.111626441s Jan 28 14:08:47.066: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 46.111869217s Jan 28 14:08:49.064: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 48.110296468s Jan 28 14:08:51.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 50.110454226s Jan 28 14:08:53.064: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Succeeded", Reason="", readiness=false. Elapsed: 52.110085503s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:08:53.064[0m Jan 28 14:08:53.065: INFO: Pod "azuredisk-volume-tester-2xcnl" satisfied condition "Succeeded or Failed" Jan 28 14:08:53.065: INFO: deleting Pod "azuredisk-4728"/"azuredisk-volume-tester-2xcnl" Jan 28 14:08:53.182: INFO: Pod azuredisk-volume-tester-2xcnl has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-2xcnl in namespace azuredisk-4728 [38;5;243m01/28/23 14:08:53.182[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:08:53.298[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:08:53.353[0m ... skipping 33 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:08:00.762[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:08:00.762[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:08:00.819[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:08:00.819[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:08:00.877[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:08:00.878[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:08:00.954[0m Jan 28 14:08:00.954: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-2xcnl" in namespace "azuredisk-4728" to be "Succeeded or Failed" Jan 28 14:08:01.009: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 54.527747ms Jan 28 14:08:03.064: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110043283s Jan 28 14:08:05.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110744221s Jan 28 14:08:07.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.111270269s Jan 28 14:08:09.066: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111909997s Jan 28 14:08:11.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110373915s ... skipping 16 lines ... Jan 28 14:08:45.066: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 44.111626441s Jan 28 14:08:47.066: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 46.111869217s Jan 28 14:08:49.064: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 48.110296468s Jan 28 14:08:51.065: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Pending", Reason="", readiness=false. Elapsed: 50.110454226s Jan 28 14:08:53.064: INFO: Pod "azuredisk-volume-tester-2xcnl": Phase="Succeeded", Reason="", readiness=false. Elapsed: 52.110085503s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:08:53.064[0m Jan 28 14:08:53.065: INFO: Pod "azuredisk-volume-tester-2xcnl" satisfied condition "Succeeded or Failed" Jan 28 14:08:53.065: INFO: deleting Pod "azuredisk-4728"/"azuredisk-volume-tester-2xcnl" Jan 28 14:08:53.182: INFO: Pod azuredisk-volume-tester-2xcnl has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-2xcnl in namespace azuredisk-4728 [38;5;243m01/28/23 14:08:53.182[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:08:53.298[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:08:53.353[0m ... skipping 34 lines ... [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:09:35.032[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:09:35.032[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:09:35.096[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:09:35.096[0m [1mSTEP:[0m checking that the pod has 'FailedMount' event [38;5;243m01/28/23 14:09:35.153[0m Jan 28 14:09:57.265: INFO: deleting Pod "azuredisk-5466"/"azuredisk-volume-tester-gbwnx" Jan 28 14:09:57.350: INFO: Error getting logs for pod azuredisk-volume-tester-gbwnx: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-gbwnx) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-gbwnx in namespace azuredisk-5466 [38;5;243m01/28/23 14:09:57.35[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:09:57.466[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:09:57.52[0m Jan 28 14:09:57.520: INFO: deleting PVC "azuredisk-5466"/"pvc-zjhcz" Jan 28 14:09:57.520: INFO: Deleting PersistentVolumeClaim "pvc-zjhcz" [1mSTEP:[0m waiting for claim's PV "pvc-40788de4-a49d-4b94-8b6c-15f280502412" to be deleted [38;5;243m01/28/23 14:09:57.576[0m ... skipping 33 lines ... [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:09:35.032[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:09:35.032[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:09:35.096[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:09:35.096[0m [1mSTEP:[0m checking that the pod has 'FailedMount' event [38;5;243m01/28/23 14:09:35.153[0m Jan 28 14:09:57.265: INFO: deleting Pod "azuredisk-5466"/"azuredisk-volume-tester-gbwnx" Jan 28 14:09:57.350: INFO: Error getting logs for pod azuredisk-volume-tester-gbwnx: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-gbwnx) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-gbwnx in namespace azuredisk-5466 [38;5;243m01/28/23 14:09:57.35[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:09:57.466[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:09:57.52[0m Jan 28 14:09:57.520: INFO: deleting PVC "azuredisk-5466"/"pvc-zjhcz" Jan 28 14:09:57.520: INFO: Deleting PersistentVolumeClaim "pvc-zjhcz" [1mSTEP:[0m waiting for claim's PV "pvc-40788de4-a49d-4b94-8b6c-15f280502412" to be deleted [38;5;243m01/28/23 14:09:57.576[0m ... skipping 30 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:10:44.178[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:10:44.179[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:10:44.234[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:10:44.234[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:10:44.294[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:10:44.294[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:10:44.35[0m Jan 28 14:10:44.350: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-q8kvg" in namespace "azuredisk-2790" to be "Succeeded or Failed" Jan 28 14:10:44.404: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 53.865246ms Jan 28 14:10:46.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109698936s Jan 28 14:10:48.458: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 4.107839982s Jan 28 14:10:50.458: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 6.108157255s Jan 28 14:10:52.461: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110838792s Jan 28 14:10:54.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109509695s ... skipping 3 lines ... Jan 28 14:11:02.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 18.110200658s Jan 28 14:11:04.459: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 20.108717359s Jan 28 14:11:06.459: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 22.109045159s Jan 28 14:11:08.459: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 24.108722021s Jan 28 14:11:10.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Succeeded", Reason="", readiness=false. Elapsed: 26.10989375s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:11:10.46[0m Jan 28 14:11:10.460: INFO: Pod "azuredisk-volume-tester-q8kvg" satisfied condition "Succeeded or Failed" Jan 28 14:11:10.460: INFO: deleting Pod "azuredisk-2790"/"azuredisk-volume-tester-q8kvg" Jan 28 14:11:10.518: INFO: Pod azuredisk-volume-tester-q8kvg has the following logs: e2e-test [1mSTEP:[0m Deleting pod azuredisk-volume-tester-q8kvg in namespace azuredisk-2790 [38;5;243m01/28/23 14:11:10.518[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:11:10.64[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:11:10.694[0m ... skipping 33 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:10:44.178[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:10:44.179[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:10:44.234[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:10:44.234[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:10:44.294[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:10:44.294[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:10:44.35[0m Jan 28 14:10:44.350: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-q8kvg" in namespace "azuredisk-2790" to be "Succeeded or Failed" Jan 28 14:10:44.404: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 53.865246ms Jan 28 14:10:46.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109698936s Jan 28 14:10:48.458: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 4.107839982s Jan 28 14:10:50.458: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 6.108157255s Jan 28 14:10:52.461: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110838792s Jan 28 14:10:54.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 10.109509695s ... skipping 3 lines ... Jan 28 14:11:02.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 18.110200658s Jan 28 14:11:04.459: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 20.108717359s Jan 28 14:11:06.459: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 22.109045159s Jan 28 14:11:08.459: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Pending", Reason="", readiness=false. Elapsed: 24.108722021s Jan 28 14:11:10.460: INFO: Pod "azuredisk-volume-tester-q8kvg": Phase="Succeeded", Reason="", readiness=false. Elapsed: 26.10989375s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:11:10.46[0m Jan 28 14:11:10.460: INFO: Pod "azuredisk-volume-tester-q8kvg" satisfied condition "Succeeded or Failed" Jan 28 14:11:10.460: INFO: deleting Pod "azuredisk-2790"/"azuredisk-volume-tester-q8kvg" Jan 28 14:11:10.518: INFO: Pod azuredisk-volume-tester-q8kvg has the following logs: e2e-test [1mSTEP:[0m Deleting pod azuredisk-volume-tester-q8kvg in namespace azuredisk-2790 [38;5;243m01/28/23 14:11:10.518[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:11:10.64[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:11:10.694[0m ... skipping 37 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602 [38;5;243m01/28/23 14:11:53.769[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:11:53.769[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:11:53.769[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:11:53.825[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:11:53.826[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:11:53.882[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:11:53.94[0m Jan 28 14:11:53.940: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-zkpt7" in namespace "azuredisk-5356" to be "Succeeded or Failed" Jan 28 14:11:53.994: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 54.142014ms Jan 28 14:11:56.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110205245s Jan 28 14:11:58.049: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.109683142s Jan 28 14:12:00.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110557672s Jan 28 14:12:02.051: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110827012s Jan 28 14:12:04.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110465064s ... skipping 2 lines ... Jan 28 14:12:10.052: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 16.111952966s Jan 28 14:12:12.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 18.109899728s Jan 28 14:12:14.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 20.110624635s Jan 28 14:12:16.049: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 22.109116549s Jan 28 14:12:18.051: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.111653311s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:12:18.051[0m Jan 28 14:12:18.052: INFO: Pod "azuredisk-volume-tester-zkpt7" satisfied condition "Succeeded or Failed" Jan 28 14:12:18.052: INFO: deleting Pod "azuredisk-5356"/"azuredisk-volume-tester-zkpt7" Jan 28 14:12:18.116: INFO: Pod azuredisk-volume-tester-zkpt7 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-zkpt7 in namespace azuredisk-5356 [38;5;243m01/28/23 14:12:18.116[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:12:18.227[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:12:18.282[0m ... skipping 37 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602 [38;5;243m01/28/23 14:11:53.769[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:11:53.769[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:11:53.769[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:11:53.825[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:11:53.826[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:11:53.882[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:11:53.94[0m Jan 28 14:11:53.940: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-zkpt7" in namespace "azuredisk-5356" to be "Succeeded or Failed" Jan 28 14:11:53.994: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 54.142014ms Jan 28 14:11:56.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110205245s Jan 28 14:11:58.049: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.109683142s Jan 28 14:12:00.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.110557672s Jan 28 14:12:02.051: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.110827012s Jan 28 14:12:04.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.110465064s ... skipping 2 lines ... Jan 28 14:12:10.052: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 16.111952966s Jan 28 14:12:12.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 18.109899728s Jan 28 14:12:14.050: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 20.110624635s Jan 28 14:12:16.049: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Pending", Reason="", readiness=false. Elapsed: 22.109116549s Jan 28 14:12:18.051: INFO: Pod "azuredisk-volume-tester-zkpt7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.111653311s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:12:18.051[0m Jan 28 14:12:18.052: INFO: Pod "azuredisk-volume-tester-zkpt7" satisfied condition "Succeeded or Failed" Jan 28 14:12:18.052: INFO: deleting Pod "azuredisk-5356"/"azuredisk-volume-tester-zkpt7" Jan 28 14:12:18.116: INFO: Pod azuredisk-volume-tester-zkpt7 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-zkpt7 in namespace azuredisk-5356 [38;5;243m01/28/23 14:12:18.116[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:12:18.227[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:12:18.282[0m ... skipping 44 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602 [38;5;243m01/28/23 14:13:17.395[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:13:17.396[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:13:17.396[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:13:17.451[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:13:17.451[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:13:17.506[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:13:17.562[0m Jan 28 14:13:17.563: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-9k4wz" in namespace "azuredisk-5194" to be "Succeeded or Failed" Jan 28 14:13:17.618: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 55.014746ms Jan 28 14:13:19.672: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109612345s Jan 28 14:13:21.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110515865s Jan 28 14:13:23.675: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 6.11253483s Jan 28 14:13:25.674: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111232395s Jan 28 14:13:27.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11080586s ... skipping 9 lines ... Jan 28 14:13:47.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 30.110285603s Jan 28 14:13:49.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 32.110349216s Jan 28 14:13:51.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 34.110605873s Jan 28 14:13:53.672: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 36.109797193s Jan 28 14:13:55.672: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.109526288s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:13:55.672[0m Jan 28 14:13:55.672: INFO: Pod "azuredisk-volume-tester-9k4wz" satisfied condition "Succeeded or Failed" Jan 28 14:13:55.672: INFO: deleting Pod "azuredisk-5194"/"azuredisk-volume-tester-9k4wz" Jan 28 14:13:55.760: INFO: Pod azuredisk-volume-tester-9k4wz has the following logs: hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-9k4wz in namespace azuredisk-5194 [38;5;243m01/28/23 14:13:55.76[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:13:55.889[0m ... skipping 57 lines ... [1mSTEP:[0m creating volume in external rg azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602 [38;5;243m01/28/23 14:13:17.395[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:13:17.396[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:13:17.396[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:13:17.451[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:13:17.451[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:13:17.506[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:13:17.562[0m Jan 28 14:13:17.563: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-9k4wz" in namespace "azuredisk-5194" to be "Succeeded or Failed" Jan 28 14:13:17.618: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 55.014746ms Jan 28 14:13:19.672: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109612345s Jan 28 14:13:21.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 4.110515865s Jan 28 14:13:23.675: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 6.11253483s Jan 28 14:13:25.674: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 8.111232395s Jan 28 14:13:27.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11080586s ... skipping 9 lines ... Jan 28 14:13:47.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 30.110285603s Jan 28 14:13:49.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 32.110349216s Jan 28 14:13:51.673: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 34.110605873s Jan 28 14:13:53.672: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Pending", Reason="", readiness=false. Elapsed: 36.109797193s Jan 28 14:13:55.672: INFO: Pod "azuredisk-volume-tester-9k4wz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.109526288s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:13:55.672[0m Jan 28 14:13:55.672: INFO: Pod "azuredisk-volume-tester-9k4wz" satisfied condition "Succeeded or Failed" Jan 28 14:13:55.672: INFO: deleting Pod "azuredisk-5194"/"azuredisk-volume-tester-9k4wz" Jan 28 14:13:55.760: INFO: Pod azuredisk-volume-tester-9k4wz has the following logs: hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-9k4wz in namespace azuredisk-5194 [38;5;243m01/28/23 14:13:55.76[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:13:55.889[0m ... skipping 47 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:16:20.599[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:16:20.6[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:16:20.66[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:16:20.661[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:16:20.723[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:16:20.724[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/28/23 14:16:20.781[0m Jan 28 14:16:20.782: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-ppfwk" in namespace "azuredisk-1353" to be "Error status code" Jan 28 14:16:20.844: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 62.672685ms Jan 28 14:16:22.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.120343552s Jan 28 14:16:24.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.120849254s Jan 28 14:16:26.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 6.120698387s Jan 28 14:16:28.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 8.121080494s Jan 28 14:16:30.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 10.120444921s ... skipping 24 lines ... Jan 28 14:17:20.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.121363104s Jan 28 14:17:22.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.120554547s Jan 28 14:17:24.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.120480397s Jan 28 14:17:26.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.120366569s Jan 28 14:17:28.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.121151426s Jan 28 14:17:30.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.121633768s Jan 28 14:17:32.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Failed", Reason="", readiness=false. Elapsed: 1m12.120346725s [1mSTEP:[0m Saw pod failure [38;5;243m01/28/23 14:17:32.902[0m Jan 28 14:17:32.902: INFO: Pod "azuredisk-volume-tester-ppfwk" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/28/23 14:17:32.902[0m Jan 28 14:17:33.001: INFO: deleting Pod "azuredisk-1353"/"azuredisk-volume-tester-ppfwk" Jan 28 14:17:33.060: INFO: Pod azuredisk-volume-tester-ppfwk has the following logs: touch: /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-ppfwk in namespace azuredisk-1353 [38;5;243m01/28/23 14:17:33.06[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:17:33.184[0m ... skipping 34 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:16:20.599[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:16:20.6[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:16:20.66[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:16:20.661[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:16:20.723[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:16:20.724[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/28/23 14:16:20.781[0m Jan 28 14:16:20.782: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-ppfwk" in namespace "azuredisk-1353" to be "Error status code" Jan 28 14:16:20.844: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 62.672685ms Jan 28 14:16:22.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.120343552s Jan 28 14:16:24.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.120849254s Jan 28 14:16:26.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 6.120698387s Jan 28 14:16:28.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 8.121080494s Jan 28 14:16:30.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 10.120444921s ... skipping 24 lines ... Jan 28 14:17:20.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.121363104s Jan 28 14:17:22.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.120554547s Jan 28 14:17:24.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.120480397s Jan 28 14:17:26.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.120366569s Jan 28 14:17:28.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.121151426s Jan 28 14:17:30.903: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.121633768s Jan 28 14:17:32.902: INFO: Pod "azuredisk-volume-tester-ppfwk": Phase="Failed", Reason="", readiness=false. Elapsed: 1m12.120346725s [1mSTEP:[0m Saw pod failure [38;5;243m01/28/23 14:17:32.902[0m Jan 28 14:17:32.902: INFO: Pod "azuredisk-volume-tester-ppfwk" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/28/23 14:17:32.902[0m Jan 28 14:17:33.001: INFO: deleting Pod "azuredisk-1353"/"azuredisk-volume-tester-ppfwk" Jan 28 14:17:33.060: INFO: Pod azuredisk-volume-tester-ppfwk has the following logs: touch: /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-ppfwk in namespace azuredisk-1353 [38;5;243m01/28/23 14:17:33.06[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:17:33.184[0m ... skipping 653 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:25:22.033[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:25:22.033[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:25:22.09[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:25:22.09[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:25:22.153[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:25:22.153[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:25:22.225[0m Jan 28 14:25:22.225: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-s7jjl" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 28 14:25:22.281: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 56.584295ms Jan 28 14:25:24.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114038745s Jan 28 14:25:26.341: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.116536445s Jan 28 14:25:28.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114306347s Jan 28 14:25:30.340: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115359744s Jan 28 14:25:32.340: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114879962s ... skipping 440 lines ... Jan 28 14:40:14.340: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.114697096s Jan 28 14:40:16.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.114017031s Jan 28 14:40:18.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.114066933s Jan 28 14:40:20.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.114035473s Jan 28 14:40:22.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.114169221s Jan 28 14:40:22.395: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.170483219s Jan 28 14:40:22.396: INFO: Unexpected error: <*pod.timeoutError | 0xc00098fbc0>: { msg: "timed out while waiting for pod azuredisk-59/azuredisk-volume-tester-s7jjl to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc000c30000>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azuredisk-volume-tester-s7jjl", GenerateName: "azuredisk-volume-tester-", ... skipping 138 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 28 14:40:22.397: FAIL: timed out while waiting for pod azuredisk-59/azuredisk-volume-tester-s7jjl to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x22517b7?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeCloningTest).Run(0xc000b13c28, {0x270bd00, 0xc000a36000}, 0x6?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_cloning_tester.go:55 +0x237 sigs.k8s.io/azuredisk-csi-driver/test/e2e.(*dynamicProvisioningTestSuite).defineTests.func14() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:569 +0x55a Jan 28 14:40:22.397: INFO: deleting Pod "azuredisk-59"/"azuredisk-volume-tester-s7jjl" Jan 28 14:40:22.493: INFO: Error getting logs for pod azuredisk-volume-tester-s7jjl: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-s7jjl) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-s7jjl in namespace azuredisk-59 [38;5;243m01/28/23 14:40:22.493[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:40:22.609[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:40:22.666[0m Jan 28 14:40:22.666: INFO: deleting PVC "azuredisk-59"/"pvc-7w4mk" Jan 28 14:40:22.666: INFO: Deleting PersistentVolumeClaim "pvc-7w4mk" [1mSTEP:[0m waiting for claim's PV "pvc-ef808b88-a3ac-4ef8-a863-902ae3488121" to be deleted [38;5;243m01/28/23 14:40:22.725[0m ... skipping 10 lines ... Jan 28 14:41:03.258: INFO: Waiting up to 5m0s for PersistentVolumeClaim azuredisk-59 to be removed Jan 28 14:41:03.314: INFO: Claim "azuredisk-59" in namespace "pvc-7w4mk" doesn't exist in the system Jan 28 14:41:03.314: INFO: deleting StorageClass azuredisk-59-disk.csi.azure.com-dynamic-sc-vkqqg [1mSTEP:[0m dump namespace information after failure [38;5;243m01/28/23 14:41:03.373[0m [1mSTEP:[0m Destroying namespace "azuredisk-59" for this suite. [38;5;243m01/28/23 14:41:03.374[0m [38;5;243m------------------------------[0m [38;5;9m• [FAILED] [942.272 seconds][0m Dynamic Provisioning [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:41[0m [multi-az] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:48[0m [38;5;9m[1m[It] should clone a volume from an existing volume and read from it [disk.csi.azure.com][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:539[0m ... skipping 8 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:25:22.033[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:25:22.033[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:25:22.09[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:25:22.09[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:25:22.153[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:25:22.153[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:25:22.225[0m Jan 28 14:25:22.225: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-s7jjl" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 28 14:25:22.281: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 56.584295ms Jan 28 14:25:24.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114038745s Jan 28 14:25:26.341: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.116536445s Jan 28 14:25:28.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114306347s Jan 28 14:25:30.340: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115359744s Jan 28 14:25:32.340: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114879962s ... skipping 440 lines ... Jan 28 14:40:14.340: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.114697096s Jan 28 14:40:16.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.114017031s Jan 28 14:40:18.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.114066933s Jan 28 14:40:20.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.114035473s Jan 28 14:40:22.339: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.114169221s Jan 28 14:40:22.395: INFO: Pod "azuredisk-volume-tester-s7jjl": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.170483219s Jan 28 14:40:22.396: INFO: Unexpected error: <*pod.timeoutError | 0xc00098fbc0>: { msg: "timed out while waiting for pod azuredisk-59/azuredisk-volume-tester-s7jjl to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc000c30000>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azuredisk-volume-tester-s7jjl", GenerateName: "azuredisk-volume-tester-", ... skipping 138 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 28 14:40:22.397: FAIL: timed out while waiting for pod azuredisk-59/azuredisk-volume-tester-s7jjl to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x22517b7?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeCloningTest).Run(0xc000b13c28, {0x270bd00, 0xc000a36000}, 0x6?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_cloning_tester.go:55 +0x237 sigs.k8s.io/azuredisk-csi-driver/test/e2e.(*dynamicProvisioningTestSuite).defineTests.func14() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:569 +0x55a Jan 28 14:40:22.397: INFO: deleting Pod "azuredisk-59"/"azuredisk-volume-tester-s7jjl" Jan 28 14:40:22.493: INFO: Error getting logs for pod azuredisk-volume-tester-s7jjl: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-s7jjl) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-s7jjl in namespace azuredisk-59 [38;5;243m01/28/23 14:40:22.493[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:40:22.609[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:40:22.666[0m Jan 28 14:40:22.666: INFO: deleting PVC "azuredisk-59"/"pvc-7w4mk" Jan 28 14:40:22.666: INFO: Deleting PersistentVolumeClaim "pvc-7w4mk" [1mSTEP:[0m waiting for claim's PV "pvc-ef808b88-a3ac-4ef8-a863-902ae3488121" to be deleted [38;5;243m01/28/23 14:40:22.725[0m ... skipping 11 lines ... Jan 28 14:41:03.314: INFO: Claim "azuredisk-59" in namespace "pvc-7w4mk" doesn't exist in the system Jan 28 14:41:03.314: INFO: deleting StorageClass azuredisk-59-disk.csi.azure.com-dynamic-sc-vkqqg [1mSTEP:[0m dump namespace information after failure [38;5;243m01/28/23 14:41:03.373[0m [1mSTEP:[0m Destroying namespace "azuredisk-59" for this suite. [38;5;243m01/28/23 14:41:03.374[0m [38;5;243m<< End Captured GinkgoWriter Output[0m [38;5;9mJan 28 14:40:22.397: timed out while waiting for pod azuredisk-59/azuredisk-volume-tester-s7jjl to be Succeeded or Failed[0m [38;5;9mIn [1m[It][0m[38;5;9m at: [1m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [1mThere were additional failures detected after the initial failure:[0m [38;5;13m[PANICKED][0m [38;5;13mTest Panicked[0m [38;5;13mIn [1m[DeferCleanup (Each)][0m[38;5;13m at: [1m/usr/local/go/src/runtime/panic.go:260[0m [38;5;13mruntime error: invalid memory address or nil pointer dereference[0m [38;5;13mFull Stack Trace[0m k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:274 +0x5c k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0000203c0) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:271 +0x179 ... skipping 18 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:41:04.345[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:41:04.345[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:41:04.403[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:41:04.403[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:41:04.462[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:41:04.462[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:41:04.521[0m Jan 28 14:41:04.521: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-4pmlc" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 28 14:41:04.577: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 56.137271ms Jan 28 14:41:06.639: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 2.117690725s Jan 28 14:41:08.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 4.113523409s Jan 28 14:41:10.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 6.113972694s Jan 28 14:41:12.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 8.114052402s Jan 28 14:41:14.636: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11505487s ... skipping 440 lines ... Jan 28 14:55:56.636: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.114879435s Jan 28 14:55:58.637: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.115386051s Jan 28 14:56:00.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.113956591s Jan 28 14:56:02.636: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.114992804s Jan 28 14:56:04.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.113457295s Jan 28 14:56:04.692: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.170892653s Jan 28 14:56:04.693: INFO: Unexpected error: <*pod.timeoutError | 0xc0008dc000>: { msg: "timed out while waiting for pod azuredisk-2546/azuredisk-volume-tester-4pmlc to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc000571b00>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azuredisk-volume-tester-4pmlc", GenerateName: "azuredisk-volume-tester-", ... skipping 139 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 28 14:56:04.693: FAIL: timed out while waiting for pod azuredisk-2546/azuredisk-volume-tester-4pmlc to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x22517b7?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeCloningTest).Run(0xc000bd1c28, {0x270bd00, 0xc000a4a340}, 0x13?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_cloning_tester.go:55 +0x237 sigs.k8s.io/azuredisk-csi-driver/test/e2e.(*dynamicProvisioningTestSuite).defineTests.func15() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:612 +0x696 Jan 28 14:56:04.694: INFO: deleting Pod "azuredisk-2546"/"azuredisk-volume-tester-4pmlc" Jan 28 14:56:04.797: INFO: Error getting logs for pod azuredisk-volume-tester-4pmlc: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-4pmlc) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-4pmlc in namespace azuredisk-2546 [38;5;243m01/28/23 14:56:04.797[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:56:04.914[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:56:04.972[0m Jan 28 14:56:04.972: INFO: deleting PVC "azuredisk-2546"/"pvc-tq5wr" Jan 28 14:56:04.972: INFO: Deleting PersistentVolumeClaim "pvc-tq5wr" [1mSTEP:[0m waiting for claim's PV "pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f" to be deleted [38;5;243m01/28/23 14:56:05.031[0m ... skipping 12 lines ... Jan 28 14:56:55.680: INFO: Waiting up to 5m0s for PersistentVolumeClaim azuredisk-2546 to be removed Jan 28 14:56:55.737: INFO: Claim "azuredisk-2546" in namespace "pvc-tq5wr" doesn't exist in the system Jan 28 14:56:55.737: INFO: deleting StorageClass azuredisk-2546-disk.csi.azure.com-dynamic-sc-rk2hh [1mSTEP:[0m dump namespace information after failure [38;5;243m01/28/23 14:56:55.796[0m [1mSTEP:[0m Destroying namespace "azuredisk-2546" for this suite. [38;5;243m01/28/23 14:56:55.796[0m [38;5;243m------------------------------[0m [38;5;9m• [FAILED] [952.421 seconds][0m Dynamic Provisioning [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:41[0m [multi-az] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:48[0m [38;5;9m[1m[It] should clone a volume of larger size than the source volume and make sure the filesystem is appropriately adjusted [disk.csi.azure.com][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:572[0m ... skipping 8 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:41:04.345[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:41:04.345[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:41:04.403[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:41:04.403[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:41:04.462[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:41:04.462[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:41:04.521[0m Jan 28 14:41:04.521: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-4pmlc" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 28 14:41:04.577: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 56.137271ms Jan 28 14:41:06.639: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 2.117690725s Jan 28 14:41:08.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 4.113523409s Jan 28 14:41:10.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 6.113972694s Jan 28 14:41:12.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 8.114052402s Jan 28 14:41:14.636: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11505487s ... skipping 440 lines ... Jan 28 14:55:56.636: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.114879435s Jan 28 14:55:58.637: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.115386051s Jan 28 14:56:00.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.113956591s Jan 28 14:56:02.636: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.114992804s Jan 28 14:56:04.635: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.113457295s Jan 28 14:56:04.692: INFO: Pod "azuredisk-volume-tester-4pmlc": Phase="Pending", Reason="", readiness=false. Elapsed: 15m0.170892653s Jan 28 14:56:04.693: INFO: Unexpected error: <*pod.timeoutError | 0xc0008dc000>: { msg: "timed out while waiting for pod azuredisk-2546/azuredisk-volume-tester-4pmlc to be Succeeded or Failed", observedObjects: [ <*v1.Pod | 0xc000571b00>{ TypeMeta: {Kind: "", APIVersion: ""}, ObjectMeta: { Name: "azuredisk-volume-tester-4pmlc", GenerateName: "azuredisk-volume-tester-", ... skipping 139 lines ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output Jan 28 14:56:04.693: FAIL: timed out while waiting for pod azuredisk-2546/azuredisk-volume-tester-4pmlc to be Succeeded or Failed Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x22517b7?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeCloningTest).Run(0xc000bd1c28, {0x270bd00, 0xc000a4a340}, 0x13?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_cloning_tester.go:55 +0x237 sigs.k8s.io/azuredisk-csi-driver/test/e2e.(*dynamicProvisioningTestSuite).defineTests.func15() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:612 +0x696 Jan 28 14:56:04.694: INFO: deleting Pod "azuredisk-2546"/"azuredisk-volume-tester-4pmlc" Jan 28 14:56:04.797: INFO: Error getting logs for pod azuredisk-volume-tester-4pmlc: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-4pmlc) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-4pmlc in namespace azuredisk-2546 [38;5;243m01/28/23 14:56:04.797[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 14:56:04.914[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 14:56:04.972[0m Jan 28 14:56:04.972: INFO: deleting PVC "azuredisk-2546"/"pvc-tq5wr" Jan 28 14:56:04.972: INFO: Deleting PersistentVolumeClaim "pvc-tq5wr" [1mSTEP:[0m waiting for claim's PV "pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f" to be deleted [38;5;243m01/28/23 14:56:05.031[0m ... skipping 13 lines ... Jan 28 14:56:55.737: INFO: Claim "azuredisk-2546" in namespace "pvc-tq5wr" doesn't exist in the system Jan 28 14:56:55.737: INFO: deleting StorageClass azuredisk-2546-disk.csi.azure.com-dynamic-sc-rk2hh [1mSTEP:[0m dump namespace information after failure [38;5;243m01/28/23 14:56:55.796[0m [1mSTEP:[0m Destroying namespace "azuredisk-2546" for this suite. [38;5;243m01/28/23 14:56:55.796[0m [38;5;243m<< End Captured GinkgoWriter Output[0m [38;5;9mJan 28 14:56:04.693: timed out while waiting for pod azuredisk-2546/azuredisk-volume-tester-4pmlc to be Succeeded or Failed[0m [38;5;9mIn [1m[It][0m[38;5;9m at: [1m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [1mThere were additional failures detected after the initial failure:[0m [38;5;13m[PANICKED][0m [38;5;13mTest Panicked[0m [38;5;13mIn [1m[DeferCleanup (Each)][0m[38;5;13m at: [1m/usr/local/go/src/runtime/panic.go:260[0m [38;5;13mruntime error: invalid memory address or nil pointer dereference[0m [38;5;13mFull Stack Trace[0m k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:274 +0x5c k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0000203c0) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:271 +0x179 ... skipping 28 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:56:56.943[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:56:56.943[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:56:57.001[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:56:57.001[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:56:57.058[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:56:57.058[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:56:57.116[0m Jan 28 14:56:57.116: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-k5bv9" in namespace "azuredisk-1598" to be "Succeeded or Failed" Jan 28 14:56:57.180: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 64.168598ms Jan 28 14:56:59.238: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.12169784s Jan 28 14:57:01.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.121531759s Jan 28 14:57:03.241: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.124856718s Jan 28 14:57:05.238: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.121774783s Jan 28 14:57:07.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.121494574s ... skipping 10 lines ... Jan 28 14:57:29.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 32.121659793s Jan 28 14:57:31.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 34.120807921s Jan 28 14:57:33.239: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 36.123526277s Jan 28 14:57:35.239: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Running", Reason="", readiness=true. Elapsed: 38.122841512s Jan 28 14:57:37.239: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.122719366s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:57:37.239[0m Jan 28 14:57:37.239: INFO: Pod "azuredisk-volume-tester-k5bv9" satisfied condition "Succeeded or Failed" Jan 28 14:57:37.239: INFO: deleting Pod "azuredisk-1598"/"azuredisk-volume-tester-k5bv9" Jan 28 14:57:37.327: INFO: Pod azuredisk-volume-tester-k5bv9 has the following logs: hello world hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-k5bv9 in namespace azuredisk-1598 [38;5;243m01/28/23 14:57:37.327[0m ... skipping 75 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:56:56.943[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:56:56.943[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:56:57.001[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:56:57.001[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:56:57.058[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:56:57.058[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:56:57.116[0m Jan 28 14:56:57.116: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-k5bv9" in namespace "azuredisk-1598" to be "Succeeded or Failed" Jan 28 14:56:57.180: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 64.168598ms Jan 28 14:56:59.238: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.12169784s Jan 28 14:57:01.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.121531759s Jan 28 14:57:03.241: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.124856718s Jan 28 14:57:05.238: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.121774783s Jan 28 14:57:07.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.121494574s ... skipping 10 lines ... Jan 28 14:57:29.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 32.121659793s Jan 28 14:57:31.237: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 34.120807921s Jan 28 14:57:33.239: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Pending", Reason="", readiness=false. Elapsed: 36.123526277s Jan 28 14:57:35.239: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Running", Reason="", readiness=true. Elapsed: 38.122841512s Jan 28 14:57:37.239: INFO: Pod "azuredisk-volume-tester-k5bv9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.122719366s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:57:37.239[0m Jan 28 14:57:37.239: INFO: Pod "azuredisk-volume-tester-k5bv9" satisfied condition "Succeeded or Failed" Jan 28 14:57:37.239: INFO: deleting Pod "azuredisk-1598"/"azuredisk-volume-tester-k5bv9" Jan 28 14:57:37.327: INFO: Pod azuredisk-volume-tester-k5bv9 has the following logs: hello world hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-k5bv9 in namespace azuredisk-1598 [38;5;243m01/28/23 14:57:37.327[0m ... skipping 69 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:59:10.559[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:59:10.559[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:59:10.617[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:59:10.617[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:59:10.675[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:59:10.675[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:59:10.733[0m Jan 28 14:59:10.733: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-582gh" in namespace "azuredisk-3410" to be "Succeeded or Failed" Jan 28 14:59:10.791: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 57.743968ms Jan 28 14:59:12.850: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 2.116393333s Jan 28 14:59:14.850: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 4.116429393s Jan 28 14:59:16.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 6.115128746s Jan 28 14:59:18.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115009558s Jan 28 14:59:20.851: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 10.117711346s ... skipping 10 lines ... Jan 28 14:59:42.850: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 32.117055081s Jan 28 14:59:44.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 34.115226136s Jan 28 14:59:46.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 36.115006079s Jan 28 14:59:48.851: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Running", Reason="", readiness=true. Elapsed: 38.118238187s Jan 28 14:59:50.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.115146935s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:59:50.848[0m Jan 28 14:59:50.849: INFO: Pod "azuredisk-volume-tester-582gh" satisfied condition "Succeeded or Failed" Jan 28 14:59:50.849: INFO: deleting Pod "azuredisk-3410"/"azuredisk-volume-tester-582gh" Jan 28 14:59:50.934: INFO: Pod azuredisk-volume-tester-582gh has the following logs: 100+0 records in 100+0 records out 104857600 bytes (100.0MB) copied, 0.084721 seconds, 1.2GB/s hello world ... skipping 53 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 14:59:10.559[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 14:59:10.559[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 14:59:10.617[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 14:59:10.617[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 14:59:10.675[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 14:59:10.675[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 14:59:10.733[0m Jan 28 14:59:10.733: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-582gh" in namespace "azuredisk-3410" to be "Succeeded or Failed" Jan 28 14:59:10.791: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 57.743968ms Jan 28 14:59:12.850: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 2.116393333s Jan 28 14:59:14.850: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 4.116429393s Jan 28 14:59:16.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 6.115128746s Jan 28 14:59:18.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115009558s Jan 28 14:59:20.851: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 10.117711346s ... skipping 10 lines ... Jan 28 14:59:42.850: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 32.117055081s Jan 28 14:59:44.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 34.115226136s Jan 28 14:59:46.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Pending", Reason="", readiness=false. Elapsed: 36.115006079s Jan 28 14:59:48.851: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Running", Reason="", readiness=true. Elapsed: 38.118238187s Jan 28 14:59:50.848: INFO: Pod "azuredisk-volume-tester-582gh": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.115146935s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 14:59:50.848[0m Jan 28 14:59:50.849: INFO: Pod "azuredisk-volume-tester-582gh" satisfied condition "Succeeded or Failed" Jan 28 14:59:50.849: INFO: deleting Pod "azuredisk-3410"/"azuredisk-volume-tester-582gh" Jan 28 14:59:50.934: INFO: Pod azuredisk-volume-tester-582gh has the following logs: 100+0 records in 100+0 records out 104857600 bytes (100.0MB) copied, 0.084721 seconds, 1.2GB/s hello world ... skipping 46 lines ... Jan 28 15:00:43.257: INFO: >>> kubeConfig: /root/tmp1936064140/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:00:43.258[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:00:43.258[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:00:43.316[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:00:43.316[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:00:43.377[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:00:43.437[0m Jan 28 15:00:43.437: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-t9t9q" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 28 15:00:43.494: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 56.636522ms Jan 28 15:00:45.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11621057s Jan 28 15:00:47.555: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.118220706s Jan 28 15:00:49.551: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114084772s Jan 28 15:00:51.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.116137798s Jan 28 15:00:53.551: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114138919s ... skipping 2 lines ... Jan 28 15:00:59.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115481459s Jan 28 15:01:01.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115527963s Jan 28 15:01:03.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 20.115842451s Jan 28 15:01:05.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 22.115720767s Jan 28 15:01:07.554: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.116448917s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:01:07.554[0m Jan 28 15:01:07.554: INFO: Pod "azuredisk-volume-tester-t9t9q" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 15:01:07.554[0m 2023/01/28 15:01:07 Running in Prow, converting AZURE_CREDENTIALS to AZURE_CREDENTIAL_FILE 2023/01/28 15:01:07 Reading credentials file /etc/azure-cred/credentials [1mSTEP:[0m Prow test resource group: kubetest-zohsaios [38;5;243m01/28/23 15:01:07.555[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-97f338d2-9f1c-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:01:07.555[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-97f338d2-9f1c-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:01:09.197[0m ... skipping 5 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:01:24.387[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:01:24.388[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:01:24.453[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:01:24.453[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 15:01:24.523[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 15:01:24.523[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:01:24.582[0m Jan 28 15:01:24.582: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-ht24x" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 28 15:01:24.640: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 57.450337ms Jan 28 15:01:26.701: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 2.118890588s Jan 28 15:01:28.699: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 4.116404018s Jan 28 15:01:30.699: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 6.117352053s Jan 28 15:01:32.697: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115322885s Jan 28 15:01:34.697: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115272236s ... skipping 2 lines ... Jan 28 15:01:40.698: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115947772s Jan 28 15:01:42.698: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115819419s Jan 28 15:01:44.701: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 20.118875027s Jan 28 15:01:46.699: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 22.116612524s Jan 28 15:01:48.697: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.114982621s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:01:48.697[0m Jan 28 15:01:48.697: INFO: Pod "azuredisk-volume-tester-ht24x" satisfied condition "Succeeded or Failed" Jan 28 15:01:48.697: INFO: deleting Pod "azuredisk-8582"/"azuredisk-volume-tester-ht24x" Jan 28 15:01:48.793: INFO: Pod azuredisk-volume-tester-ht24x has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-ht24x in namespace azuredisk-8582 [38;5;243m01/28/23 15:01:48.793[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:01:48.916[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:01:48.973[0m ... skipping 50 lines ... Jan 28 15:00:43.257: INFO: >>> kubeConfig: /root/tmp1936064140/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:00:43.258[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:00:43.258[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:00:43.316[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:00:43.316[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:00:43.377[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:00:43.437[0m Jan 28 15:00:43.437: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-t9t9q" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 28 15:00:43.494: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 56.636522ms Jan 28 15:00:45.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11621057s Jan 28 15:00:47.555: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.118220706s Jan 28 15:00:49.551: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114084772s Jan 28 15:00:51.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.116137798s Jan 28 15:00:53.551: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114138919s ... skipping 2 lines ... Jan 28 15:00:59.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115481459s Jan 28 15:01:01.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115527963s Jan 28 15:01:03.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 20.115842451s Jan 28 15:01:05.553: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Pending", Reason="", readiness=false. Elapsed: 22.115720767s Jan 28 15:01:07.554: INFO: Pod "azuredisk-volume-tester-t9t9q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.116448917s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:01:07.554[0m Jan 28 15:01:07.554: INFO: Pod "azuredisk-volume-tester-t9t9q" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 15:01:07.554[0m [1mSTEP:[0m Prow test resource group: kubetest-zohsaios [38;5;243m01/28/23 15:01:07.555[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-97f338d2-9f1c-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:01:07.555[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-97f338d2-9f1c-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:01:09.197[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/28/23 15:01:09.198[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/28/23 15:01:09.198[0m ... skipping 3 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:01:24.387[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:01:24.388[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:01:24.453[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:01:24.453[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 15:01:24.523[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 15:01:24.523[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:01:24.582[0m Jan 28 15:01:24.582: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-ht24x" in namespace "azuredisk-8582" to be "Succeeded or Failed" Jan 28 15:01:24.640: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 57.450337ms Jan 28 15:01:26.701: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 2.118890588s Jan 28 15:01:28.699: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 4.116404018s Jan 28 15:01:30.699: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 6.117352053s Jan 28 15:01:32.697: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115322885s Jan 28 15:01:34.697: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115272236s ... skipping 2 lines ... Jan 28 15:01:40.698: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115947772s Jan 28 15:01:42.698: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115819419s Jan 28 15:01:44.701: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 20.118875027s Jan 28 15:01:46.699: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Pending", Reason="", readiness=false. Elapsed: 22.116612524s Jan 28 15:01:48.697: INFO: Pod "azuredisk-volume-tester-ht24x": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.114982621s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:01:48.697[0m Jan 28 15:01:48.697: INFO: Pod "azuredisk-volume-tester-ht24x" satisfied condition "Succeeded or Failed" Jan 28 15:01:48.697: INFO: deleting Pod "azuredisk-8582"/"azuredisk-volume-tester-ht24x" Jan 28 15:01:48.793: INFO: Pod azuredisk-volume-tester-ht24x has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-ht24x in namespace azuredisk-8582 [38;5;243m01/28/23 15:01:48.793[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:01:48.916[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:01:48.973[0m ... skipping 49 lines ... Jan 28 15:03:58.252: INFO: >>> kubeConfig: /root/tmp1936064140/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:03:58.253[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:03:58.253[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:03:58.312[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:03:58.312[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:03:58.372[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:03:58.431[0m Jan 28 15:03:58.431: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-st9cf" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 28 15:03:58.487: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 56.130364ms Jan 28 15:04:00.544: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113103531s Jan 28 15:04:02.546: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114709941s Jan 28 15:04:04.545: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 6.11324208s Jan 28 15:04:06.547: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 8.11582814s Jan 28 15:04:08.547: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115826256s ... skipping 3 lines ... Jan 28 15:04:16.545: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114154335s Jan 28 15:04:18.546: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 20.115176139s Jan 28 15:04:20.550: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 22.11879642s Jan 28 15:04:22.547: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Running", Reason="", readiness=true. Elapsed: 24.11524666s Jan 28 15:04:24.546: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 26.114341865s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:04:24.546[0m Jan 28 15:04:24.546: INFO: Pod "azuredisk-volume-tester-st9cf" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 15:04:24.546[0m 2023/01/28 15:04:24 Running in Prow, converting AZURE_CREDENTIALS to AZURE_CREDENTIAL_FILE 2023/01/28 15:04:24 Reading credentials file /etc/azure-cred/credentials [1mSTEP:[0m Prow test resource group: kubetest-zohsaios [38;5;243m01/28/23 15:04:24.546[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-0d5dc85d-9f1d-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:04:24.547[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-0d5dc85d-9f1d-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:04:25.353[0m ... skipping 14 lines ... [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:04:46.704[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:04:46.762[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:04:46.762[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 15:04:46.826[0m [1mSTEP:[0m Set pod anti-affinity to make sure two pods are scheduled on different nodes [38;5;243m01/28/23 15:04:46.826[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 15:04:46.826[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:04:46.884[0m Jan 28 15:04:46.885: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-zd45g" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 28 15:04:46.941: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 56.038976ms Jan 28 15:04:48.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114348648s Jan 28 15:04:50.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114656845s Jan 28 15:04:52.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114688165s Jan 28 15:04:54.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 8.114072691s Jan 28 15:04:56.998: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 10.113528306s ... skipping 2 lines ... Jan 28 15:05:02.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 16.114527165s Jan 28 15:05:05.000: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115115577s Jan 28 15:05:06.998: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 20.11333087s Jan 28 15:05:08.998: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 22.113549333s Jan 28 15:05:11.000: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.115470949s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:05:11[0m Jan 28 15:05:11.000: INFO: Pod "azuredisk-volume-tester-zd45g" satisfied condition "Succeeded or Failed" Jan 28 15:05:11.000: INFO: deleting Pod "azuredisk-7726"/"azuredisk-volume-tester-zd45g" Jan 28 15:05:11.092: INFO: Pod azuredisk-volume-tester-zd45g has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-zd45g in namespace azuredisk-7726 [38;5;243m01/28/23 15:05:11.092[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:05:11.219[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:05:11.285[0m ... skipping 53 lines ... Jan 28 15:03:58.252: INFO: >>> kubeConfig: /root/tmp1936064140/kubeconfig/kubeconfig.westus2.json [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:03:58.253[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:03:58.253[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:03:58.312[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:03:58.312[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:03:58.372[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:03:58.431[0m Jan 28 15:03:58.431: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-st9cf" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 28 15:03:58.487: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 56.130364ms Jan 28 15:04:00.544: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113103531s Jan 28 15:04:02.546: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114709941s Jan 28 15:04:04.545: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 6.11324208s Jan 28 15:04:06.547: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 8.11582814s Jan 28 15:04:08.547: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 10.115826256s ... skipping 3 lines ... Jan 28 15:04:16.545: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114154335s Jan 28 15:04:18.546: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 20.115176139s Jan 28 15:04:20.550: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Pending", Reason="", readiness=false. Elapsed: 22.11879642s Jan 28 15:04:22.547: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Running", Reason="", readiness=true. Elapsed: 24.11524666s Jan 28 15:04:24.546: INFO: Pod "azuredisk-volume-tester-st9cf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 26.114341865s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:04:24.546[0m Jan 28 15:04:24.546: INFO: Pod "azuredisk-volume-tester-st9cf" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 15:04:24.546[0m [1mSTEP:[0m Prow test resource group: kubetest-zohsaios [38;5;243m01/28/23 15:04:24.546[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-0d5dc85d-9f1d-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:04:24.547[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-0d5dc85d-9f1d-11ed-89a5-fa733b842602 [38;5;243m01/28/23 15:04:25.353[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/28/23 15:04:25.353[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/28/23 15:04:25.353[0m ... skipping 12 lines ... [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:04:46.704[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:04:46.762[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:04:46.762[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 15:04:46.826[0m [1mSTEP:[0m Set pod anti-affinity to make sure two pods are scheduled on different nodes [38;5;243m01/28/23 15:04:46.826[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 15:04:46.826[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:04:46.884[0m Jan 28 15:04:46.885: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-zd45g" in namespace "azuredisk-7726" to be "Succeeded or Failed" Jan 28 15:04:46.941: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 56.038976ms Jan 28 15:04:48.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114348648s Jan 28 15:04:50.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114656845s Jan 28 15:04:52.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114688165s Jan 28 15:04:54.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 8.114072691s Jan 28 15:04:56.998: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 10.113528306s ... skipping 2 lines ... Jan 28 15:05:02.999: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 16.114527165s Jan 28 15:05:05.000: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 18.115115577s Jan 28 15:05:06.998: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 20.11333087s Jan 28 15:05:08.998: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Pending", Reason="", readiness=false. Elapsed: 22.113549333s Jan 28 15:05:11.000: INFO: Pod "azuredisk-volume-tester-zd45g": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.115470949s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:05:11[0m Jan 28 15:05:11.000: INFO: Pod "azuredisk-volume-tester-zd45g" satisfied condition "Succeeded or Failed" Jan 28 15:05:11.000: INFO: deleting Pod "azuredisk-7726"/"azuredisk-volume-tester-zd45g" Jan 28 15:05:11.092: INFO: Pod azuredisk-volume-tester-zd45g has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-zd45g in namespace azuredisk-7726 [38;5;243m01/28/23 15:05:11.092[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:05:11.219[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:05:11.285[0m ... skipping 60 lines ... [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:07:20.305[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:07:20.364[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:07:20.364[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:07:20.421[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:07:20.421[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:07:20.482[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:07:20.542[0m Jan 28 15:07:20.542: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-r6jkw" in namespace "azuredisk-3086" to be "Succeeded or Failed" Jan 28 15:07:20.598: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 56.509305ms Jan 28 15:07:22.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114790704s Jan 28 15:07:24.656: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114299242s Jan 28 15:07:26.656: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 6.113899025s Jan 28 15:07:28.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115172313s Jan 28 15:07:30.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114617458s ... skipping 10 lines ... Jan 28 15:07:52.658: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 32.115933504s Jan 28 15:07:54.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 34.114931624s Jan 28 15:07:56.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 36.115433652s Jan 28 15:07:58.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 38.11466089s Jan 28 15:08:00.659: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.116739137s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:08:00.659[0m Jan 28 15:08:00.659: INFO: Pod "azuredisk-volume-tester-r6jkw" satisfied condition "Succeeded or Failed" Jan 28 15:08:00.659: INFO: deleting Pod "azuredisk-3086"/"azuredisk-volume-tester-r6jkw" Jan 28 15:08:00.724: INFO: Pod azuredisk-volume-tester-r6jkw has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-r6jkw in namespace azuredisk-3086 [38;5;243m01/28/23 15:08:00.724[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:08:00.894[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:08:00.954[0m ... skipping 70 lines ... [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:07:20.305[0m [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:07:20.364[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:07:20.364[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:07:20.421[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:07:20.421[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:07:20.482[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:07:20.542[0m Jan 28 15:07:20.542: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-r6jkw" in namespace "azuredisk-3086" to be "Succeeded or Failed" Jan 28 15:07:20.598: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 56.509305ms Jan 28 15:07:22.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.114790704s Jan 28 15:07:24.656: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114299242s Jan 28 15:07:26.656: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 6.113899025s Jan 28 15:07:28.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 8.115172313s Jan 28 15:07:30.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 10.114617458s ... skipping 10 lines ... Jan 28 15:07:52.658: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 32.115933504s Jan 28 15:07:54.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 34.114931624s Jan 28 15:07:56.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 36.115433652s Jan 28 15:07:58.657: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Pending", Reason="", readiness=false. Elapsed: 38.11466089s Jan 28 15:08:00.659: INFO: Pod "azuredisk-volume-tester-r6jkw": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.116739137s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:08:00.659[0m Jan 28 15:08:00.659: INFO: Pod "azuredisk-volume-tester-r6jkw" satisfied condition "Succeeded or Failed" Jan 28 15:08:00.659: INFO: deleting Pod "azuredisk-3086"/"azuredisk-volume-tester-r6jkw" Jan 28 15:08:00.724: INFO: Pod azuredisk-volume-tester-r6jkw has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-r6jkw in namespace azuredisk-3086 [38;5;243m01/28/23 15:08:00.724[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:08:00.894[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:08:00.954[0m ... skipping 1046 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:23:34.149[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:23:34.149[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:23:34.208[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:23:34.208[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 15:23:34.268[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:23:34.268[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:23:34.327[0m Jan 28 15:23:34.327: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-rfbnf" in namespace "azuredisk-1092" to be "Succeeded or Failed" Jan 28 15:23:34.383: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 56.208497ms Jan 28 15:23:36.441: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113762314s Jan 28 15:23:38.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114627975s Jan 28 15:23:40.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114676629s Jan 28 15:23:42.447: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 8.120071793s Jan 28 15:23:44.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11488524s ... skipping 2 lines ... Jan 28 15:23:50.443: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115671561s Jan 28 15:23:52.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114628037s Jan 28 15:23:54.443: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 20.116264012s Jan 28 15:23:56.441: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 22.113726928s Jan 28 15:23:58.443: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.115404269s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:23:58.443[0m Jan 28 15:23:58.443: INFO: Pod "azuredisk-volume-tester-rfbnf" satisfied condition "Succeeded or Failed" Jan 28 15:23:58.443: INFO: deleting Pod "azuredisk-1092"/"azuredisk-volume-tester-rfbnf" Jan 28 15:23:58.553: INFO: Pod azuredisk-volume-tester-rfbnf has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-rfbnf in namespace azuredisk-1092 [38;5;243m01/28/23 15:23:58.553[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:23:58.678[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:23:58.739[0m ... skipping 33 lines ... [1mSTEP:[0m setting up the StorageClass [38;5;243m01/28/23 15:23:34.149[0m [1mSTEP:[0m creating a StorageClass [38;5;243m01/28/23 15:23:34.149[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 15:23:34.208[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 15:23:34.208[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 15:23:34.268[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 15:23:34.268[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 15:23:34.327[0m Jan 28 15:23:34.327: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-rfbnf" in namespace "azuredisk-1092" to be "Succeeded or Failed" Jan 28 15:23:34.383: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 56.208497ms Jan 28 15:23:36.441: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.113762314s Jan 28 15:23:38.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114627975s Jan 28 15:23:40.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 6.114676629s Jan 28 15:23:42.447: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 8.120071793s Jan 28 15:23:44.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 10.11488524s ... skipping 2 lines ... Jan 28 15:23:50.443: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 16.115671561s Jan 28 15:23:52.442: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 18.114628037s Jan 28 15:23:54.443: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 20.116264012s Jan 28 15:23:56.441: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Pending", Reason="", readiness=false. Elapsed: 22.113726928s Jan 28 15:23:58.443: INFO: Pod "azuredisk-volume-tester-rfbnf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.115404269s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 15:23:58.443[0m Jan 28 15:23:58.443: INFO: Pod "azuredisk-volume-tester-rfbnf" satisfied condition "Succeeded or Failed" Jan 28 15:23:58.443: INFO: deleting Pod "azuredisk-1092"/"azuredisk-volume-tester-rfbnf" Jan 28 15:23:58.553: INFO: Pod azuredisk-volume-tester-rfbnf has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-rfbnf in namespace azuredisk-1092 [38;5;243m01/28/23 15:23:58.553[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 15:23:58.678[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 15:23:58.739[0m ... skipping 93 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 14:05:28.573620 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-15a7100a0e7ccd86606f3f693bffc000489534a7 e2e-test I0128 14:05:28.574227 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 14:05:28.608347 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 14:05:28.608369 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 14:05:28.608377 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 14:05:28.608408 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 14:05:28.609231 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 14:05:28.609278 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 14:05:28.609306 1 azure.go:776] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 25 lines ... I0128 14:05:28.609654 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0128 14:05:28.609676 1 azure_vmasclient.go:70] Azure AvailabilitySetsClient (read ops) using rate limit config: QPS=6, bucket=20 I0128 14:05:28.609683 1 azure_vmasclient.go:73] Azure AvailabilitySetsClient (write ops) using rate limit config: QPS=100, bucket=1000 I0128 14:05:28.609767 1 azure.go:1007] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0128 14:05:28.609788 1 azuredisk.go:192] disable UseInstanceMetadata for controller I0128 14:05:28.609796 1 azuredisk.go:204] cloud: AzurePublicCloud, location: westus2, rg: kubetest-zohsaios, VMType: vmss, PrimaryScaleSetName: k8s-agentpool-15504031-vmss, PrimaryAvailabilitySetName: , DisableAvailabilitySetNodes: false I0128 14:05:28.612521 1 mount_linux.go:287] 'umount /tmp/kubelet-detect-safe-umount3144104350' failed with: exit status 32, output: umount: /tmp/kubelet-detect-safe-umount3144104350: must be superuser to unmount. I0128 14:05:28.612550 1 mount_linux.go:289] Detected umount with unsafe 'not mounted' behavior I0128 14:05:28.612634 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0128 14:05:28.612643 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0128 14:05:28.612650 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0128 14:05:28.612657 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0128 14:05:28.612663 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 68 lines ... I0128 14:05:41.752475 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 24989 I0128 14:05:41.841434 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 32358 I0128 14:05:41.845298 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:05:41.845358 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb to node k8s-agentpool-15504031-vmss000001 I0128 14:05:41.845406 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb:%!s(*provider.AttachDiskOptions=&{None pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb false 0})] I0128 14:05:41.845450 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb:%!s(*provider.AttachDiskOptions=&{None pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb false 0})]) I0128 14:05:42.554364 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb:%!s(*provider.AttachDiskOptions=&{None pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:05:52.660236 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:05:52.660284 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:05:52.660305 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb attached to node k8s-agentpool-15504031-vmss000001. I0128 14:05:52.660318 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:05:52.660363 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=11.001940734 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:05:52.660381 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 18 lines ... I0128 14:06:47.175055 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb) returned with <nil> I0128 14:06:47.175093 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.186975839 request="azuredisk_csi_driver_controller_delete_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-5f6c3009-e2fe-41de-b717-40c161f4cdeb" result_code="succeeded" I0128 14:06:47.175109 1 utils.go:84] GRPC response: {} I0128 14:06:52.541179 1 utils.go:77] GRPC call: /csi.v1.Controller/CreateVolume I0128 14:06:52.541289 1 utils.go:78] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}},{"segments":{"topology.disk.csi.azure.com/zone":"westus2-1","topology.kubernetes.io/zone":"westus2-1"}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-1","topology.kubernetes.io/zone":"westus2-1"}},{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633","csi.storage.k8s.io/pvc/name":"pvc-m4bmj","csi.storage.k8s.io/pvc/namespace":"azuredisk-2540","enableAsyncAttach":"false","networkAccessPolicy":"DenyAll","skuName":"Standard_LRS","userAgent":"azuredisk-e2e-test"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]} I0128 14:06:52.543108 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 14:06:52.548359 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 14:06:52.548581 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 14:06:52.548747 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 14:06:52.548982 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 14:06:52.549776 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 14:06:52.549863 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 14:06:52.549888 1 azure.go:776] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 37 lines ... I0128 14:06:56.892253 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633","csi.storage.k8s.io/pvc/name":"pvc-m4bmj","csi.storage.k8s.io/pvc/namespace":"azuredisk-2540","enableAsyncAttach":"false","enableasyncattach":"false","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com","userAgent":"azuredisk-e2e-test"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633"} I0128 14:06:56.912825 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1217 I0128 14:06:56.913248 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:06:56.913278 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 to node k8s-agentpool-15504031-vmss000001 I0128 14:06:56.913336 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 false 0})] I0128 14:06:56.913404 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 false 0})]) I0128 14:06:57.059523 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:07:07.159121 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:07:07.159164 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:07:07.159186 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:07:07.159387 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:07:07.159467 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.246187324 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:07:07.159507 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 18 lines ... I0128 14:07:55.442913 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633) returned with <nil> I0128 14:07:55.442957 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.266002571 request="azuredisk_csi_driver_controller_delete_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-c1893ec8-d9d6-4569-9fa3-fda8fb6e3633" result_code="succeeded" I0128 14:07:55.442978 1 utils.go:84] GRPC response: {} I0128 14:08:00.924248 1 utils.go:77] GRPC call: /csi.v1.Controller/CreateVolume I0128 14:08:00.924270 1 utils.go:78] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":"westus2-2","topology.kubernetes.io/zone":"westus2-2"}}]},"capacity_range":{"required_bytes":1099511627776},"name":"pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36","csi.storage.k8s.io/pvc/name":"pvc-q8xrb","csi.storage.k8s.io/pvc/namespace":"azuredisk-4728","enableAsyncAttach":"false","enableBursting":"true","perfProfile":"Basic","skuName":"Premium_LRS","userAgent":"azuredisk-e2e-test"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]} I0128 14:08:00.924816 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 14:08:00.933769 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 14:08:00.933797 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 14:08:00.933806 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 14:08:00.933838 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 14:08:00.934236 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 14:08:00.934282 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 14:08:00.934298 1 azure.go:776] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 37 lines ... I0128 14:08:03.992172 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36","csi.storage.k8s.io/pvc/name":"pvc-q8xrb","csi.storage.k8s.io/pvc/namespace":"azuredisk-4728","enableAsyncAttach":"false","enableBursting":"true","enableasyncattach":"false","perfProfile":"Basic","requestedsizegib":"1024","skuName":"Premium_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com","userAgent":"azuredisk-e2e-test"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36"} I0128 14:08:04.014222 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1338 I0128 14:08:04.014552 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:08:04.014588 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 to node k8s-agentpool-15504031-vmss000001 I0128 14:08:04.014625 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 false 0})] I0128 14:08:04.014692 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 false 0})]) I0128 14:08:04.138552 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:08:14.282994 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:08:14.283030 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:08:14.283052 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:08:14.283067 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:08:14.283108 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.268566063 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d9c690a4-3c8c-4990-b885-af4ee1a1bc36" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:08:14.283125 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 32 lines ... I0128 14:09:38.187721 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-40788de4-a49d-4b94-8b6c-15f280502412","csi.storage.k8s.io/pvc/name":"pvc-zjhcz","csi.storage.k8s.io/pvc/namespace":"azuredisk-5466","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412"} I0128 14:09:38.219720 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 14:09:38.220036 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-40788de4-a49d-4b94-8b6c-15f280502412. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:09:38.220071 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412 to node k8s-agentpool-15504031-vmss000001 I0128 14:09:38.220109 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-40788de4-a49d-4b94-8b6c-15f280502412 false 0})] I0128 14:09:38.220187 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-40788de4-a49d-4b94-8b6c-15f280502412 false 0})]) I0128 14:09:38.414171 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-40788de4-a49d-4b94-8b6c-15f280502412 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:09:48.517410 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:09:48.517446 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:09:48.517468 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:09:48.517482 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:09:48.517524 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.297492378 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:09:48.517543 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 40 lines ... I0128 14:10:47.384230 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2","csi.storage.k8s.io/pvc/name":"pvc-6bnmk","csi.storage.k8s.io/pvc/namespace":"azuredisk-2790","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2"} I0128 14:10:47.405092 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 14:10:47.405321 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:10:47.405349 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 to node k8s-agentpool-15504031-vmss000001 I0128 14:10:47.405388 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 false 0})] I0128 14:10:47.405435 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 false 0})]) I0128 14:10:47.624227 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:10:57.705917 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:10:57.705956 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:10:57.705976 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:10:57.705992 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:10:57.706032 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.300705739 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:10:57.706048 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 45 lines ... I0128 14:11:56.993465 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c","csi.storage.k8s.io/pvc/name":"pvc-9cjjm","csi.storage.k8s.io/pvc/namespace":"azuredisk-5356","requestedsizegib":"10","resourceGroup":"azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c"} I0128 14:11:57.024193 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1238 I0128 14:11:57.024479 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:11:57.024518 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c to node k8s-agentpool-15504031-vmss000001 I0128 14:11:57.024552 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c false 0})] I0128 14:11:57.024593 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c false 0})]) I0128 14:11:57.314501 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:12:07.501111 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:12:07.501152 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:12:07.501176 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c attached to node k8s-agentpool-15504031-vmss000001. I0128 14:12:07.501193 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:12:07.501242 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.476759999 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:12:07.501267 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 55 lines ... I0128 14:13:20.633204 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806 to node k8s-agentpool-15504031-vmss000001 I0128 14:13:20.633246 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fac3f81-702b-4922-b310-351ca80e1806 false 0})] I0128 14:13:20.633320 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fac3f81-702b-4922-b310-351ca80e1806 false 0})]) I0128 14:13:20.668269 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1238 I0128 14:13:20.668744 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-20f6a35b-0112-4361-aa1b-192768fbae48. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:13:20.668783 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48 to node k8s-agentpool-15504031-vmss000001 I0128 14:13:21.551493 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9fac3f81-702b-4922-b310-351ca80e1806 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:13:31.700693 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:13:31.700726 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:13:31.700757 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:13:31.700867 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:13:31.700791 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48 lun 1 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-20f6a35b-0112-4361-aa1b-192768fbae48 false 1})] I0128 14:13:31.700914 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=11.067737809 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8745872-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-9fac3f81-702b-4922-b310-351ca80e1806" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:13:31.700933 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-20f6a35b-0112-4361-aa1b-192768fbae48 false 1})]) I0128 14:13:31.700929 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0128 14:13:31.879256 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/microsoft.compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-20f6a35b-0112-4361-aa1b-192768fbae48 false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:13:42.059771 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:13:42.059819 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:13:42.059843 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:13:42.059863 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:13:42.059938 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=21.39117881 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-e8f3b9f9-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-20f6a35b-0112-4361-aa1b-192768fbae48" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:13:42.059956 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 54 lines ... I0128 14:16:23.845440 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1207 I0128 14:16:23.887311 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 24989 I0128 14:16:23.896604 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:16:23.896642 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b to node k8s-agentpool-15504031-vmss000001 I0128 14:16:23.896679 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b false 0})] I0128 14:16:23.896722 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b false 0})]) I0128 14:16:24.765803 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:16:59.986436 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:16:59.986695 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:16:59.986815 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b attached to node k8s-agentpool-15504031-vmss000001. I0128 14:16:59.986891 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:16:59.987023 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=36.141122384 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-afa634d6-804e-4bd4-8f8b-510a61b8d46b" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:16:59.987107 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 40 lines ... I0128 14:18:18.125363 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150","csi.storage.k8s.io/pvc/name":"pvc-cr4xg","csi.storage.k8s.io/pvc/namespace":"azuredisk-2888","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150"} I0128 14:18:18.149158 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 14:18:18.149513 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:18:18.149548 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 to node k8s-agentpool-15504031-vmss000001 I0128 14:18:18.149587 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 false 0})] I0128 14:18:18.149625 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 false 0})]) I0128 14:18:18.295689 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:18:28.410090 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:18:28.410130 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:18:28.410412 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:18:28.410442 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:18:28.410513 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.260970751 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cb3bd3bc-cb13-4511-b761-f3b75bd18150" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:18:28.410533 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 11 lines ... I0128 14:18:42.499733 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000000","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee","csi.storage.k8s.io/pvc/name":"pvc-64rfk","csi.storage.k8s.io/pvc/namespace":"azuredisk-2888","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee"} I0128 14:18:42.549119 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 14:18:42.549440 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee to node k8s-agentpool-15504031-vmss000000 (vmState Succeeded). I0128 14:18:42.549498 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee to node k8s-agentpool-15504031-vmss000000 I0128 14:18:42.549540 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee lun 0 to node k8s-agentpool-15504031-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee false 0})] I0128 14:18:42.549584 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee false 0})]) I0128 14:18:42.707002 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:18:52.815623 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000000) successfully I0128 14:18:52.815658 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000000) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:18:52.815679 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee attached to node k8s-agentpool-15504031-vmss000000. I0128 14:18:52.815693 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee to node k8s-agentpool-15504031-vmss000000 successfully I0128 14:18:52.815737 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.266302267 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-956ad2b4-e70a-4242-9c5e-2c91efd21eee" node="k8s-agentpool-15504031-vmss000000" result_code="succeeded" I0128 14:18:52.815761 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 11 lines ... I0128 14:19:06.787382 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e","csi.storage.k8s.io/pvc/name":"pvc-bblw7","csi.storage.k8s.io/pvc/namespace":"azuredisk-2888","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e"} I0128 14:19:06.810625 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 14:19:06.811000 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:19:06.811030 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e to node k8s-agentpool-15504031-vmss000001 I0128 14:19:06.811063 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e lun 1 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e false 1})] I0128 14:19:06.811111 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e false 1})]) I0128 14:19:07.027753 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:19:17.184884 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:19:17.184936 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:19:17.184964 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e attached to node k8s-agentpool-15504031-vmss000001. I0128 14:19:17.184984 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:19:17.185055 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.374028822 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-f5add3cf-2e07-48c3-8bb8-665273e4bd6e" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:19:17.185073 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 75 lines ... I0128 14:22:53.885710 1 azure_vmss_cache.go:327] refresh the cache of NonVmssUniformNodesCache in rg map[kubetest-zohsaios:{}] I0128 14:22:53.911589 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 12 I0128 14:22:53.911674 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:22:53.911704 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 to node k8s-agentpool-15504031-vmss000001 I0128 14:22:53.911745 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 false 0})] I0128 14:22:53.911775 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 false 0})]) I0128 14:22:54.048639 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:23:04.143530 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:23:04.143570 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:23:04.143593 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:23:04.143610 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:23:04.143655 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.257932056 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-af75fa0d-c2ed-45a2-9d35-caa1dbff7c74" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:23:04.143685 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 57 lines ... I0128 14:25:25.339166 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:25:25.466348 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1217 I0128 14:25:25.466699 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-ef808b88-a3ac-4ef8-a863-902ae3488121. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:25:25.466737 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 to node k8s-agentpool-15504031-vmss000001 I0128 14:25:25.466777 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 false 0})] I0128 14:25:25.466819 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 false 0})]) I0128 14:25:25.703964 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:25:35.891008 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:25:35.891060 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:25:35.891095 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:25:35.891119 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:25:35.891181 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.424502354 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:25:35.891215 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 36 lines ... I0128 14:41:07.556092 1 azure_vmss_cache.go:327] refresh the cache of NonVmssUniformNodesCache in rg map[kubetest-zohsaios:{}] I0128 14:41:07.579005 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 12 I0128 14:41:07.579100 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:41:07.579169 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f to node k8s-agentpool-15504031-vmss000001 I0128 14:41:07.579214 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f false 0})] I0128 14:41:07.579334 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f false 0})]) I0128 14:41:07.707994 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:41:17.833849 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:41:17.833886 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:41:17.833907 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f attached to node k8s-agentpool-15504031-vmss000001. I0128 14:41:17.833920 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:41:17.833961 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.277852515 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:41:17.834027 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 66 lines ... I0128 14:57:00.222883 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a6e434b-866e-4971-a669-6b64223306b2 false 0})] I0128 14:57:00.222963 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a6e434b-866e-4971-a669-6b64223306b2 false 0})]) I0128 14:57:00.222632 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-a2bcaab9-e610-4048-8655-fed9d7a24364. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a2bcaab9-e610-4048-8655-fed9d7a24364 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:57:00.223110 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a2bcaab9-e610-4048-8655-fed9d7a24364 to node k8s-agentpool-15504031-vmss000001 I0128 14:57:00.222685 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-750ce9d2-7e3d-4c6b-88b1-4cc3080f7ee3. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-750ce9d2-7e3d-4c6b-88b1-4cc3080f7ee3 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:57:00.223188 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-750ce9d2-7e3d-4c6b-88b1-4cc3080f7ee3 to node k8s-agentpool-15504031-vmss000001 I0128 14:57:00.353521 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2a6e434b-866e-4971-a669-6b64223306b2 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:57:10.462524 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:57:10.462560 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:57:10.462768 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:57:10.462796 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:57:10.462863 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.266205047 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:57:10.462881 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 4 lines ... I0128 14:57:10.502379 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1421 I0128 14:57:10.502818 1 azure_controller_common.go:516] azureDisk - find disk: lun 0 name pvc-2a6e434b-866e-4971-a669-6b64223306b2 uri /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2 I0128 14:57:10.502843 1 controllerserver.go:383] GetDiskLun returned: <nil>. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:57:10.502860 1 controllerserver.go:398] Attach operation is successful. volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2 is already attached to node k8s-agentpool-15504031-vmss000001 at lun 0. I0128 14:57:10.503121 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.000291302 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:57:10.503145 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0128 14:57:10.642727 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-750ce9d2-7e3d-4c6b-88b1-4cc3080f7ee3:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-750ce9d2-7e3d-4c6b-88b1-4cc3080f7ee3 false 2}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-a2bcaab9-e610-4048-8655-fed9d7a24364:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-a2bcaab9-e610-4048-8655-fed9d7a24364 false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:57:20.750885 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:57:20.750926 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:57:20.751122 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a2bcaab9-e610-4048-8655-fed9d7a24364 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:57:20.751139 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-750ce9d2-7e3d-4c6b-88b1-4cc3080f7ee3 lun 2 to node k8s-agentpool-15504031-vmss000001, diskMap: map[] I0128 14:57:20.751149 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a2bcaab9-e610-4048-8655-fed9d7a24364 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:57:20.751176 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-750ce9d2-7e3d-4c6b-88b1-4cc3080f7ee3 attached to node k8s-agentpool-15504031-vmss000001. ... skipping 98 lines ... I0128 14:59:13.905872 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 to node k8s-agentpool-15504031-vmss000001 I0128 14:59:13.905906 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-81b06a61-b121-48d7-9355-3b86f4171a35 false 0})] I0128 14:59:13.905965 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-81b06a61-b121-48d7-9355-3b86f4171a35 false 0})]) I0128 14:59:13.911069 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1191 I0128 14:59:13.911354 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 14:59:13.911385 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a to node k8s-agentpool-15504031-vmss000001 I0128 14:59:14.678779 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-81b06a61-b121-48d7-9355-3b86f4171a35 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:59:24.770683 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:59:24.770720 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:59:24.770748 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 attached to node k8s-agentpool-15504031-vmss000001. I0128 14:59:24.770766 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:59:24.770812 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.864957627 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:59:24.770833 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0128 14:59:24.770923 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a lun 1 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a false 1})] I0128 14:59:24.771005 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a false 1})]) I0128 14:59:24.903631 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 14:59:35.028808 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 14:59:35.028845 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 14:59:35.028865 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a attached to node k8s-agentpool-15504031-vmss000001. I0128 14:59:35.028879 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a to node k8s-agentpool-15504031-vmss000001 successfully I0128 14:59:35.028921 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=21.117562584 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-d645f4fa-d373-4c1f-bd25-f8b5b38c3b7a" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 14:59:35.028942 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 53 lines ... I0128 15:00:46.443868 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3d9450a0-a354-44cc-9143-1e628a4374e5","csi.storage.k8s.io/pvc/name":"pvc-jcdtv","csi.storage.k8s.io/pvc/namespace":"azuredisk-8582","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5"} I0128 15:00:46.475114 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 15:00:46.475440 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-3d9450a0-a354-44cc-9143-1e628a4374e5. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:00:46.475472 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 to node k8s-agentpool-15504031-vmss000001 I0128 15:00:46.475505 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 false 0})] I0128 15:00:46.475575 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 false 0})]) I0128 15:00:46.741982 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:00:56.838416 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:00:56.838458 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:00:56.838484 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:00:56.838501 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:00:56.838547 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.363124573 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:00:56.838630 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 37 lines ... I0128 15:01:32.033692 1 utils.go:84] GRPC response: {} I0128 15:01:32.033806 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-70480c61-5740-483c-a970-56fd0b5cc06a lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-70480c61-5740-483c-a970-56fd0b5cc06a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-70480c61-5740-483c-a970-56fd0b5cc06a false 0})] I0128 15:01:32.033841 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-70480c61-5740-483c-a970-56fd0b5cc06a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-70480c61-5740-483c-a970-56fd0b5cc06a false 0})]) I0128 15:01:32.062320 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume I0128 15:01:32.062345 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5"} I0128 15:01:32.062536 1 controllerserver.go:471] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5 from node k8s-agentpool-15504031-vmss000001 I0128 15:01:32.173596 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-70480c61-5740-483c-a970-56fd0b5cc06a:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-70480c61-5740-483c-a970-56fd0b5cc06a false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:01:42.262765 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:01:42.262797 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:01:42.262830 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-70480c61-5740-483c-a970-56fd0b5cc06a attached to node k8s-agentpool-15504031-vmss000001. I0128 15:01:42.262846 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-70480c61-5740-483c-a970-56fd0b5cc06a to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:01:42.262900 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=14.582024063 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-70480c61-5740-483c-a970-56fd0b5cc06a" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:01:42.262921 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 56 lines ... I0128 15:04:01.443555 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-14d182da-4219-4eea-af6a-ad4ef063b90b","csi.storage.k8s.io/pvc/name":"pvc-8s92h","csi.storage.k8s.io/pvc/namespace":"azuredisk-7726","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b"} I0128 15:04:01.470427 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 15:04:01.470843 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-14d182da-4219-4eea-af6a-ad4ef063b90b. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:04:01.470875 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b to node k8s-agentpool-15504031-vmss000001 I0128 15:04:01.470936 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-14d182da-4219-4eea-af6a-ad4ef063b90b false 0})] I0128 15:04:01.471005 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-14d182da-4219-4eea-af6a-ad4ef063b90b false 0})]) I0128 15:04:01.666417 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-14d182da-4219-4eea-af6a-ad4ef063b90b false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:04:11.866361 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:04:11.866394 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:04:11.866414 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b attached to node k8s-agentpool-15504031-vmss000001. I0128 15:04:11.866428 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:04:11.866469 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.395626048 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-14d182da-4219-4eea-af6a-ad4ef063b90b" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:04:11.866488 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 29 lines ... I0128 15:04:49.988022 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000000","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f","csi.storage.k8s.io/pvc/name":"pvc-4m2nf","csi.storage.k8s.io/pvc/namespace":"azuredisk-7726","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f"} I0128 15:04:50.010025 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1501 I0128 15:04:50.010377 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f to node k8s-agentpool-15504031-vmss000000 (vmState Succeeded). I0128 15:04:50.010405 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f to node k8s-agentpool-15504031-vmss000000 I0128 15:04:50.010440 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f lun 0 to node k8s-agentpool-15504031-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f false 0})] I0128 15:04:50.010474 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f false 0})]) I0128 15:04:50.168560 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:05:00.276282 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000000) successfully I0128 15:05:00.276316 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000000) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:05:00.276336 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f attached to node k8s-agentpool-15504031-vmss000000. I0128 15:05:00.276351 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f to node k8s-agentpool-15504031-vmss000000 successfully I0128 15:05:00.276391 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.266016739 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ec3aa37d-99c9-41c5-a3ff-9da10e6adf7f" node="k8s-agentpool-15504031-vmss000000" result_code="succeeded" I0128 15:05:00.276411 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 87 lines ... I0128 15:07:23.683089 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1221 I0128 15:07:23.683425 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-16d6ef1f-6cad-45ae-9efb-69d749546343. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:07:23.683452 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 to node k8s-agentpool-15504031-vmss000001 I0128 15:07:23.684560 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1219 I0128 15:07:23.684839 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:07:23.684866 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62 to node k8s-agentpool-15504031-vmss000001 I0128 15:07:25.639934 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-cf84cddc-facf-461b-970b-0352a3029a8f:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-cf84cddc-facf-461b-970b-0352a3029a8f false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:07:35.724067 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:07:35.724112 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:07:35.724146 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cf84cddc-facf-461b-970b-0352a3029a8f attached to node k8s-agentpool-15504031-vmss000001. I0128 15:07:35.724163 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cf84cddc-facf-461b-970b-0352a3029a8f to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:07:35.724239 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=12.047715957 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-cf84cddc-facf-461b-970b-0352a3029a8f" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:07:35.724358 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} I0128 15:07:35.725463 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 lun 1 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62 false 2})] I0128 15:07:35.725566 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62 false 2})]) I0128 15:07:35.878022 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 false 1}) /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2cb88024-e53a-44e0-a287-05a6f2f96a62 false 2})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:07:45.977318 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:07:45.977351 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:07:45.977380 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:07:45.977396 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:07:45.977437 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=22.294010788 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-16d6ef1f-6cad-45ae-9efb-69d749546343" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:07:45.977459 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 74 lines ... I0128 15:09:37.137838 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-4dppw-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1387","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d"} I0128 15:09:37.160911 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0128 15:09:37.161297 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:09:37.161330 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d to node k8s-agentpool-15504031-vmss000001 I0128 15:09:37.161387 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d false 0})] I0128 15:09:37.161480 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d false 0})]) I0128 15:09:38.808817 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:09:43.947827 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:09:43.947870 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:09:43.947897 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d attached to node k8s-agentpool-15504031-vmss000001. I0128 15:09:43.947917 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:09:43.947973 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=6.786665279 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:09:43.948003 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 77 lines ... I0128 15:12:21.821972 1 azure_vmss_cache.go:327] refresh the cache of NonVmssUniformNodesCache in rg map[kubetest-zohsaios:{}] I0128 15:12:21.845501 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 12 I0128 15:12:21.845750 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:12:21.845837 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d to node k8s-agentpool-15504031-vmss000001 I0128 15:12:21.845986 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d false 0})] I0128 15:12:21.846036 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d false 0})]) I0128 15:12:22.006105 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:12:32.103917 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:12:32.103960 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:12:32.103982 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d attached to node k8s-agentpool-15504031-vmss000001. I0128 15:12:32.103996 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:12:32.104035 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.28204933 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:12:32.104056 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 19 lines ... I0128 15:12:58.086263 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000000","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-4703b58c-5198-402e-8e28-98f95b067150","csi.storage.k8s.io/pvc/name":"pvc-f67j9","csi.storage.k8s.io/pvc/namespace":"azuredisk-4801","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com","tags":"disk=test"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150"} I0128 15:12:58.112029 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1214 I0128 15:12:58.112322 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-4703b58c-5198-402e-8e28-98f95b067150. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150 to node k8s-agentpool-15504031-vmss000000 (vmState Succeeded). I0128 15:12:58.112352 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150 to node k8s-agentpool-15504031-vmss000000 I0128 15:12:58.112385 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150 lun 0 to node k8s-agentpool-15504031-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4703b58c-5198-402e-8e28-98f95b067150 false 0})] I0128 15:12:58.112422 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4703b58c-5198-402e-8e28-98f95b067150 false 0})]) I0128 15:12:58.275968 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-4703b58c-5198-402e-8e28-98f95b067150 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:13:08.379467 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000000) successfully I0128 15:13:08.379503 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000000) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:13:08.379526 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150 attached to node k8s-agentpool-15504031-vmss000000. I0128 15:13:08.379544 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150 to node k8s-agentpool-15504031-vmss000000 successfully I0128 15:13:08.379991 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.267266669 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-4703b58c-5198-402e-8e28-98f95b067150" node="k8s-agentpool-15504031-vmss000000" result_code="succeeded" I0128 15:13:08.380012 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 16 lines ... I0128 15:13:42.837942 1 azure_controller_common.go:398] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d from node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d] E0128 15:13:42.837979 1 azure_controller_vmss.go:202] detach azure disk on node(k8s-agentpool-15504031-vmss000001): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d]) not found I0128 15:13:42.837996 1 azure_controller_vmss.go:239] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d]) I0128 15:13:43.108594 1 utils.go:77] GRPC call: /csi.v1.Controller/DeleteVolume I0128 15:13:43.108865 1 utils.go:78] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d"} I0128 15:13:43.108986 1 controllerserver.go:317] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d) I0128 15:13:43.109006 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d) since it's in attaching or detaching state I0128 15:13:43.109139 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=4.0801e-05 request="azuredisk_csi_driver_controller_delete_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d" result_code="failed_csi_driver_controller_delete_volume" E0128 15:13:43.109193 1 utils.go:82] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d) since it's in attaching or detaching state I0128 15:13:48.014563 1 azure_controller_vmss.go:252] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - detach disk(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d:pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d]) returned with <nil> I0128 15:13:48.014630 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:13:48.014685 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:13:48.014703 1 azure_controller_common.go:422] azureDisk - detach disk(pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d) succeeded I0128 15:13:48.014724 1 controllerserver.go:480] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d from node k8s-agentpool-15504031-vmss000001 successfully I0128 15:13:48.014765 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.176925675 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ccf6f8f7-c384-4e5c-a06b-b17de4aa1e8d" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" ... skipping 41 lines ... I0128 15:14:30.193075 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca","csi.storage.k8s.io/pvc/name":"pvc-9bp5s","csi.storage.k8s.io/pvc/namespace":"azuredisk-8154","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca"} I0128 15:14:30.218504 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1193 I0128 15:14:30.219019 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:14:30.219060 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca to node k8s-agentpool-15504031-vmss000001 I0128 15:14:30.219136 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca false 0})] I0128 15:14:30.219248 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca false 0})]) I0128 15:14:30.459444 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:14:40.590937 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:14:40.590990 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:14:40.591021 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca attached to node k8s-agentpool-15504031-vmss000001. I0128 15:14:40.591044 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:14:40.591105 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.372080958 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-de1104e8-4a4f-49b5-9da3-2e83709b09ca" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:14:40.591143 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 44 lines ... I0128 15:16:07.195101 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-6m762-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1166","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060"} I0128 15:16:07.216996 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0128 15:16:07.217389 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:16:07.217419 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 to node k8s-agentpool-15504031-vmss000001 I0128 15:16:07.217451 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 false 0})] I0128 15:16:07.217489 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 false 0})]) I0128 15:16:07.399508 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:16:17.488713 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:16:17.488755 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:16:17.488779 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:16:17.488795 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:16:17.488842 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.271442404 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-066f7ae7-0d9f-49ca-94af-6ee5949c5060" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:16:17.488866 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 12 lines ... I0128 15:17:37.121634 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1192 I0128 15:17:37.162768 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 24989 I0128 15:17:37.166889 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-0f5bf240-37a7-47de-9060-20071163ce09. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 to node k8s-agentpool-15504031-vmss000000 (vmState Succeeded). I0128 15:17:37.166936 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 to node k8s-agentpool-15504031-vmss000000 I0128 15:17:37.166997 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 lun 0 to node k8s-agentpool-15504031-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0f5bf240-37a7-47de-9060-20071163ce09 false 0})] I0128 15:17:37.167046 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0f5bf240-37a7-47de-9060-20071163ce09 false 0})]) I0128 15:17:37.327181 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0f5bf240-37a7-47de-9060-20071163ce09 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:17:47.458782 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000000) successfully I0128 15:17:47.458819 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000000) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:17:47.458839 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 attached to node k8s-agentpool-15504031-vmss000000. I0128 15:17:47.458853 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 to node k8s-agentpool-15504031-vmss000000 successfully I0128 15:17:47.458895 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.336976374 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09" node="k8s-agentpool-15504031-vmss000000" result_code="succeeded" I0128 15:17:47.458914 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 48 lines ... I0128 15:18:53.261009 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-0f5bf240-37a7-47de-9060-20071163ce09","csi.storage.k8s.io/pvc/name":"pvc-z6d5w","csi.storage.k8s.io/pvc/namespace":"azuredisk-783","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09"} I0128 15:18:53.286335 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1192 I0128 15:18:53.286607 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-0f5bf240-37a7-47de-9060-20071163ce09. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:18:53.286640 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 to node k8s-agentpool-15504031-vmss000001 I0128 15:18:53.286693 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0f5bf240-37a7-47de-9060-20071163ce09 false 0})] I0128 15:18:53.286726 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0f5bf240-37a7-47de-9060-20071163ce09 false 0})]) I0128 15:18:53.445656 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-0f5bf240-37a7-47de-9060-20071163ce09 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:19:03.544586 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:19:03.544701 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:19:03.544744 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:19:03.544777 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:19:03.544841 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.25823469 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-0f5bf240-37a7-47de-9060-20071163ce09" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:19:03.544860 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 32 lines ... I0128 15:21:31.318117 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":5}},"volume_context":{"cachingmode":"None","csi.storage.k8s.io/pv/name":"pvc-07b723c6-491f-4faa-96d7-28d557aded76","csi.storage.k8s.io/pvc/name":"pvc-htvj8","csi.storage.k8s.io/pvc/namespace":"azuredisk-7920","maxshares":"2","requestedsizegib":"10","skuname":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76"} I0128 15:21:31.339599 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1214 I0128 15:21:31.340012 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-07b723c6-491f-4faa-96d7-28d557aded76. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:21:31.340043 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 to node k8s-agentpool-15504031-vmss000001 I0128 15:21:31.340209 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76:%!s(*provider.AttachDiskOptions=&{None pvc-07b723c6-491f-4faa-96d7-28d557aded76 false 0})] I0128 15:21:31.340275 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76:%!s(*provider.AttachDiskOptions=&{None pvc-07b723c6-491f-4faa-96d7-28d557aded76 false 0})]) I0128 15:21:31.536940 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76:%!s(*provider.AttachDiskOptions=&{None pvc-07b723c6-491f-4faa-96d7-28d557aded76 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:21:32.105001 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerPublishVolume I0128 15:21:32.105026 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000000","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":5}},"volume_context":{"cachingmode":"None","csi.storage.k8s.io/pv/name":"pvc-07b723c6-491f-4faa-96d7-28d557aded76","csi.storage.k8s.io/pvc/name":"pvc-htvj8","csi.storage.k8s.io/pvc/namespace":"azuredisk-7920","maxshares":"2","requestedsizegib":"10","skuname":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76"} I0128 15:21:32.132235 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1692 I0128 15:21:32.132718 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-07b723c6-491f-4faa-96d7-28d557aded76. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 to node k8s-agentpool-15504031-vmss000000 (vmState Succeeded). I0128 15:21:32.132752 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 to node k8s-agentpool-15504031-vmss000000 I0128 15:21:32.132817 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 lun 0 to node k8s-agentpool-15504031-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76:%!s(*provider.AttachDiskOptions=&{None pvc-07b723c6-491f-4faa-96d7-28d557aded76 false 0})] I0128 15:21:32.132912 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76:%!s(*provider.AttachDiskOptions=&{None pvc-07b723c6-491f-4faa-96d7-28d557aded76 false 0})]) I0128 15:21:32.306550 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76:%!s(*provider.AttachDiskOptions=&{None pvc-07b723c6-491f-4faa-96d7-28d557aded76 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:21:51.728055 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:21:51.728092 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:21:51.728108 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:21:51.728118 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:21:51.728165 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=20.388143664 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:21:51.728177 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 50 lines ... I0128 15:23:37.365113 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-6db6d396-e290-4392-91a4-4ccca47155e7","csi.storage.k8s.io/pvc/name":"pvc-p5l4t","csi.storage.k8s.io/pvc/namespace":"azuredisk-1092","device-setting/device/queue_depth":"17","device-setting/queue/max_sectors_kb":"211","device-setting/queue/nr_requests":"44","device-setting/queue/read_ahead_kb":"256","device-setting/queue/rotational":"0","device-setting/queue/scheduler":"none","device-setting/queue/wbt_lat_usec":"0","perfProfile":"advanced","requestedsizegib":"10","skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7"} I0128 15:23:37.392114 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1222 I0128 15:23:37.392531 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-6db6d396-e290-4392-91a4-4ccca47155e7. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:23:37.392566 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7 to node k8s-agentpool-15504031-vmss000001 I0128 15:23:37.392737 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6db6d396-e290-4392-91a4-4ccca47155e7 false 0})] I0128 15:23:37.392787 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6db6d396-e290-4392-91a4-4ccca47155e7 false 0})]) I0128 15:23:37.576598 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-6db6d396-e290-4392-91a4-4ccca47155e7 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:23:47.653875 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:23:47.653911 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:23:47.653933 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:23:47.653947 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:23:47.653990 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.261455309 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:23:47.654013 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 31 lines ... I0128 15:24:44.749319 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697"} I0128 15:24:44.770227 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1219 I0128 15:24:44.770725 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:24:44.770762 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 to node k8s-agentpool-15504031-vmss000001 I0128 15:24:44.770800 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 lun 0 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 false 0})] I0128 15:24:44.770874 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 false 0})]) I0128 15:24:45.019098 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:24:55.185430 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:24:55.185486 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:24:55.185518 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:24:55.185542 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:24:55.185815 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.414904333 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ab10be8b-8270-4ad0-9f8d-19d6c0f17697" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:24:55.185855 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 18 lines ... I0128 15:25:09.719523 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000000","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9"} I0128 15:25:09.742931 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1248 I0128 15:25:09.743216 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 to node k8s-agentpool-15504031-vmss000000 (vmState Succeeded). I0128 15:25:09.743251 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 to node k8s-agentpool-15504031-vmss000000 I0128 15:25:09.743285 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 lun 0 to node k8s-agentpool-15504031-vmss000000, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 false 0})] I0128 15:25:09.743324 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 false 0})]) I0128 15:25:09.881552 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000000) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 false 0})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:25:20.045810 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000000) successfully I0128 15:25:20.045846 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000000) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:25:20.045867 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 attached to node k8s-agentpool-15504031-vmss000000. I0128 15:25:20.045883 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9 to node k8s-agentpool-15504031-vmss000000 successfully I0128 15:25:20.045923 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.302710595 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9" node="k8s-agentpool-15504031-vmss000000" result_code="succeeded" I0128 15:25:20.045974 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"0"}} ... skipping 10 lines ... I0128 15:25:35.813582 1 utils.go:78] GRPC request: {"node_id":"k8s-agentpool-15504031-vmss000001","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-nonroot-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7"} I0128 15:25:35.834347 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1256 I0128 15:25:35.834720 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 to node k8s-agentpool-15504031-vmss000001 (vmState Succeeded). I0128 15:25:35.834756 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 to node k8s-agentpool-15504031-vmss000001 I0128 15:25:35.834793 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 lun 1 to node k8s-agentpool-15504031-vmss000001, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 false 1})] I0128 15:25:35.834839 1 azure_controller_vmss.go:110] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 false 1})]) I0128 15:25:36.089034 1 azure_controller_vmss.go:122] azureDisk - update(kubetest-zohsaios): vm(k8s-agentpool-15504031-vmss000001) - attach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-zohsaios/providers/microsoft.compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7:%!s(*provider.AttachDiskOptions=&{ReadOnly pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 false 1})], %!s(*retry.Error=<nil>)) returned with %!v(MISSING) I0128 15:25:46.190505 1 azure_vmss_cache.go:275] DeleteCacheForNode(kubetest-zohsaios, k8s-agentpool-15504031-vmss, k8s-agentpool-15504031-vmss000001) successfully I0128 15:25:46.190556 1 azure_vmss_cache.go:313] updateCache(k8s-agentpool-15504031-vmss, kubetest-zohsaios, k8s-agentpool-15504031-vmss000001) for cacheKey(kubetest-zohsaios/k8s-agentpool-15504031-vmss) updated successfully I0128 15:25:46.190580 1 controllerserver.go:413] Attach operation successful: volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 attached to node k8s-agentpool-15504031-vmss000001. I0128 15:25:46.190597 1 controllerserver.go:433] attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7 to node k8s-agentpool-15504031-vmss000001 successfully I0128 15:25:46.190643 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=10.355925275 request="azuredisk_csi_driver_controller_publish_volume" resource_group="kubetest-zohsaios" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2e9915c5-fa22-45a2-9cb7-4b80c105dcb7" node="k8s-agentpool-15504031-vmss000001" result_code="succeeded" I0128 15:25:46.190694 1 utils.go:84] GRPC response: {"publish_context":{"LUN":"1"}} ... skipping 12 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 14:05:34.343340 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-15a7100a0e7ccd86606f3f693bffc000489534a7 e2e-test I0128 14:05:34.344819 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 14:05:34.374414 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 14:05:34.374483 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 14:05:34.374494 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 14:05:34.374547 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 14:05:34.375595 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 14:05:34.375702 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 14:05:34.375738 1 azure.go:776] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 25 lines ... I0128 14:05:34.376273 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0128 14:05:34.376297 1 azure_vmasclient.go:70] Azure AvailabilitySetsClient (read ops) using rate limit config: QPS=6, bucket=20 I0128 14:05:34.376310 1 azure_vmasclient.go:73] Azure AvailabilitySetsClient (write ops) using rate limit config: QPS=100, bucket=1000 I0128 14:05:34.376414 1 azure.go:1007] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0128 14:05:34.376450 1 azuredisk.go:192] disable UseInstanceMetadata for controller I0128 14:05:34.376462 1 azuredisk.go:204] cloud: AzurePublicCloud, location: westus2, rg: kubetest-zohsaios, VMType: vmss, PrimaryScaleSetName: k8s-agentpool-15504031-vmss, PrimaryAvailabilitySetName: , DisableAvailabilitySetNodes: false I0128 14:05:34.380320 1 mount_linux.go:287] 'umount /tmp/kubelet-detect-safe-umount4217029496' failed with: exit status 32, output: umount: /tmp/kubelet-detect-safe-umount4217029496: must be superuser to unmount. I0128 14:05:34.380366 1 mount_linux.go:289] Detected umount with unsafe 'not mounted' behavior I0128 14:05:34.380554 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0128 14:05:34.380685 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0128 14:05:34.380780 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0128 14:05:34.380887 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0128 14:05:34.380980 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 62 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 14:05:29.400425 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-15a7100a0e7ccd86606f3f693bffc000489534a7 e2e-test I0128 14:05:29.401095 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 14:05:29.430995 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 14:05:29.431028 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 14:05:29.431037 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 14:05:29.431067 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 14:05:29.431924 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 14:05:29.431957 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 14:05:29.431988 1 azure.go:776] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 188 lines ... I0128 14:09:55.793695 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:09:55.817415 1 mount_linux.go:570] Output: "" I0128 14:09:55.817462 1 mount_linux.go:529] Disk "/dev/disk/azure/scsi1/lun0" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/scsi1/lun0] I0128 14:09:56.301866 1 mount_linux.go:539] Disk successfully formatted (mkfs): ext4 - /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount I0128 14:09:56.301901 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount I0128 14:09:56.301929 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount) E0128 14:09:56.320765 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:09:56.320853 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:09:56.895357 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:09:56.895389 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-40788de4-a49d-4b94-8b6c-15f280502412","csi.storage.k8s.io/pvc/name":"pvc-zjhcz","csi.storage.k8s.io/pvc/namespace":"azuredisk-5466","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412"} I0128 14:09:58.724343 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:09:58.724394 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:09:58.724898 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount with mount options([invalid mount options]) I0128 14:09:58.724928 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:09:58.735222 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 14:09:58.735271 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:09:58.749752 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount I0128 14:09:58.749828 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount) E0128 14:09:58.767023 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:09:58.767096 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:09:59.826092 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:09:59.826132 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-40788de4-a49d-4b94-8b6c-15f280502412","csi.storage.k8s.io/pvc/name":"pvc-zjhcz","csi.storage.k8s.io/pvc/namespace":"azuredisk-5466","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412"} I0128 14:10:01.588159 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:10:01.588222 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:10:01.588642 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount with mount options([invalid mount options]) I0128 14:10:01.588666 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:10:01.598473 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 14:10:01.598508 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:10:01.613248 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount I0128 14:10:01.613357 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount) E0128 14:10:01.632145 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:10:01.632202 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:10:03.688549 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:10:03.688578 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-40788de4-a49d-4b94-8b6c-15f280502412","csi.storage.k8s.io/pvc/name":"pvc-zjhcz","csi.storage.k8s.io/pvc/namespace":"azuredisk-5466","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-40788de4-a49d-4b94-8b6c-15f280502412"} I0128 14:10:05.480472 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:10:05.480527 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:10:05.480926 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount with mount options([invalid mount options]) I0128 14:10:05.480953 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:10:05.490258 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 14:10:05.490298 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:10:05.504840 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount I0128 14:10:05.504883 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount) E0128 14:10:05.521546 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:10:05.521607 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-40788de4-a49d-4b94-8b6c-15f280502412/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:11:03.204814 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:11:03.204845 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2","csi.storage.k8s.io/pvc/name":"pvc-6bnmk","csi.storage.k8s.io/pvc/namespace":"azuredisk-2790","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2"} I0128 14:11:05.088621 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:11:05.088671 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:11:05.088689 1 utils.go:84] GRPC response: {} I0128 14:11:05.097567 1 utils.go:77] GRPC call: /csi.v1.Node/NodePublishVolume ... skipping 16 lines ... I0128 14:11:10.892861 1 utils.go:84] GRPC response: {} I0128 14:11:10.921359 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 14:11:10.921390 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2"} I0128 14:11:10.921483 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 I0128 14:11:10.921514 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 14:11:10.921539 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 I0128 14:11:10.927022 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 I0128 14:11:10.927048 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2" I0128 14:11:10.927150 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-7c43015b-d011-419a-a2c8-d2bffcf2bdf2 successfully I0128 14:11:10.927166 1 utils.go:84] GRPC response: {} I0128 14:12:12.834059 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:12:12.834091 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c","csi.storage.k8s.io/pvc/name":"pvc-9cjjm","csi.storage.k8s.io/pvc/namespace":"azuredisk-5356","requestedsizegib":"10","resourceGroup":"azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-b67bb669-9f15-11ed-89a5-fa733b842602/providers/Microsoft.Compute/disks/pvc-1f12f9d3-9eb5-490b-9f87-b80603a90a8c"} I0128 14:12:14.608873 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 304 lines ... I0128 14:25:42.976592 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:25:42.991344 1 mount_linux.go:570] Output: "" I0128 14:25:42.991395 1 mount_linux.go:529] Disk "/dev/disk/azure/scsi1/lun0" appears to be unformatted, attempting to format as type: "xfs" with options: [-f /dev/disk/azure/scsi1/lun0] I0128 14:25:43.842037 1 mount_linux.go:539] Disk successfully formatted (mkfs): xfs - /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:25:43.842071 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:25:43.842096 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:25:43.869525 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:25:43.869594 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:25:44.394250 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:25:44.394287 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:25:46.260004 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:25:46.260094 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:25:46.260760 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:25:46.260810 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:25:46.273554 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:25:46.273850 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:25:46.281564 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:25:46.281615 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:25:46.292479 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:25:46.292546 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:25:47.336640 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:25:47.336670 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:25:49.182325 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:25:49.182389 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:25:49.182864 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:25:49.182897 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:25:49.191035 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:25:49.191065 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:25:49.200865 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:25:49.200905 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:25:49.214537 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:25:49.214595 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:25:51.287999 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:25:51.288030 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:25:53.127774 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:25:53.127828 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:25:53.128280 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:25:53.128309 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:25:53.137264 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:25:53.137298 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:25:53.147339 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:25:53.147392 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:25:53.161537 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:25:53.161594 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:25:57.245391 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:25:57.245422 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:25:59.105768 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:25:59.105824 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:25:59.106277 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:25:59.106306 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:25:59.114820 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:25:59.114855 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:25:59.123919 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:25:59.123957 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:25:59.135193 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:25:59.135247 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:26:07.230659 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:26:07.230714 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:26:09.121611 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:26:09.121653 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:26:09.122063 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:26:09.122079 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:26:09.134930 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:26:09.134965 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:26:09.143628 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:26:09.143674 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:26:09.155494 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:26:09.155556 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:26:25.236688 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:26:25.236722 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:26:27.032872 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:26:27.032930 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:26:27.033407 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:26:27.033442 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:26:27.043168 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:26:27.043205 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:26:27.051384 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:26:27.051435 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:26:27.071588 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:26:27.071665 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:26:59.174835 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:26:59.174880 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:27:01.067852 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:27:01.067912 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:27:01.068325 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:27:01.068341 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:27:01.079772 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:27:01.079933 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:27:01.092867 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:27:01.092916 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:27:01.107742 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:27:01.107797 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:28:05.166561 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:28:05.166591 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:28:06.976528 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:28:06.976568 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:28:06.976900 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:28:06.976916 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:28:06.989166 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:28:06.989203 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:28:06.999236 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:28:06.999285 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:28:07.010544 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:28:07.010608 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:30:09.074592 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:30:09.074631 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:30:10.859014 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:30:10.859064 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:30:10.859772 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:30:10.859800 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:30:10.869106 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:30:10.869148 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:30:10.879446 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:30:10.879494 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:30:10.891395 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:30:10.891468 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:32:12.921958 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:32:12.921996 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:32:14.745537 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:32:14.745595 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:32:14.746030 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:32:14.746045 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:32:14.757218 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:32:14.757254 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:32:14.768481 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:32:14.768544 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:32:14.781128 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:32:14.781188 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:34:16.816222 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:34:16.816254 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:34:18.674648 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:34:18.674735 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:34:18.675036 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:34:18.675053 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:34:18.687581 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:34:18.687617 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:34:18.696879 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:34:18.696925 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:34:18.708730 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:34:18.708781 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:36:20.812717 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:36:20.812756 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:36:22.710360 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:36:22.710403 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:36:22.710756 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:36:22.710773 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:36:22.733751 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:36:22.733789 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:36:22.744226 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:36:22.744272 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:36:22.754427 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:36:22.754535 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:38:24.847692 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:38:24.847721 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef808b88-a3ac-4ef8-a863-902ae3488121","csi.storage.k8s.io/pvc/name":"pvc-7w4mk","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","fsType":"xfs","requestedsizegib":"10","skuName":"Standard_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121"} I0128 14:38:26.615956 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:38:26.616006 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType Standard_LRS I0128 14:38:26.616450 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount with mount options([nouuid]) I0128 14:38:26.616477 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:38:26.626197 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:38:26.626236 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:38:26.639589 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount I0128 14:38:26.639633 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount) E0128 14:38:26.656847 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:38:26.656902 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-ef808b88-a3ac-4ef8-a863-902ae3488121/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:41:23.400062 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:41:23.400100 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:41:25.230362 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:41:25.230471 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:41:25.231169 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:41:25.231199 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:41:25.243239 1 mount_linux.go:570] Output: "" I0128 14:41:25.243278 1 mount_linux.go:529] Disk "/dev/disk/azure/scsi1/lun0" appears to be unformatted, attempting to format as type: "xfs" with options: [-f /dev/disk/azure/scsi1/lun0] I0128 14:41:25.780921 1 mount_linux.go:539] Disk successfully formatted (mkfs): xfs - /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:41:25.781306 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:41:25.781521 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:41:25.802400 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:41:25.802653 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:41:26.323527 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:41:26.323556 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:41:28.101299 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:41:28.101355 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:41:28.101867 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:41:28.101903 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:41:28.111095 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:41:28.111129 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:41:28.125509 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:41:28.125554 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:41:28.148401 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:41:28.148461 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:41:29.173050 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:41:29.173078 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:41:30.949073 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:41:30.949132 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:41:30.949509 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:41:30.949526 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:41:30.961959 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:41:30.962094 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:41:30.971184 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:41:30.971230 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:41:30.982596 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:41:30.982649 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:41:33.021999 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:41:33.022030 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:41:34.790245 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:41:34.790299 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:41:34.790954 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:41:34.790986 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:41:34.798373 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:41:34.798400 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:41:34.805963 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:41:34.806016 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:41:34.817770 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:41:34.817828 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:41:38.870565 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:41:38.870594 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:41:40.673269 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:41:40.673313 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:41:40.673728 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:41:40.673759 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:41:40.681597 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:41:40.681630 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:41:40.688638 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:41:40.688685 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:41:40.698774 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:41:40.698827 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:41:48.807488 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:41:48.807524 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:41:50.575079 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:41:50.575130 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:41:50.575454 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:41:50.575471 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:41:50.590411 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:41:50.590460 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:41:50.601130 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:41:50.601178 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:41:50.614009 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:41:50.614104 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:42:06.666043 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:42:06.666073 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:42:08.525165 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:42:08.525203 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:42:08.525625 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:42:08.525640 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:42:08.537194 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:42:08.537231 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:42:08.547353 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:42:08.547395 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:42:08.558263 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:42:08.558327 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:42:40.594758 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:42:40.594787 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:42:42.540171 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:42:42.540299 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:42:42.540748 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:42:42.540773 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:42:42.560846 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:42:42.561036 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:42:42.574473 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:42:42.574520 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:42:42.587380 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:42:42.587430 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:43:46.668274 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:43:46.668306 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:43:48.491121 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:43:48.491164 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:43:48.491526 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:43:48.491549 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:43:48.501633 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:43:48.501667 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:43:48.512457 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:43:48.512500 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:43:48.527620 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:43:48.527679 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:45:50.617262 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:45:50.617292 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:45:52.492494 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:45:52.492628 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:45:52.493500 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:45:52.493568 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:45:52.519162 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:45:52.519194 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:45:52.535563 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:45:52.535620 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:45:52.561208 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:45:52.561267 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:47:54.623770 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:47:54.623816 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:47:56.458235 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:47:56.458285 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:47:56.459749 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:47:56.459783 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:47:56.468874 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:47:56.468900 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:47:56.480069 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:47:56.480105 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:47:56.494197 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:47:56.494247 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:49:58.543141 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:49:58.543173 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:50:00.372429 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:50:00.372485 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:50:00.372998 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:50:00.373027 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:50:00.384377 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:50:00.384410 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:50:00.393284 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:50:00.393325 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:50:00.408454 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:50:00.408509 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:52:02.445066 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:52:02.445100 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:52:04.241770 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:52:04.241825 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:52:04.243758 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:52:04.243805 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:52:04.253596 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:52:04.254000 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:52:04.261952 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:52:04.262290 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:52:04.277762 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:52:04.277828 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:54:06.344500 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:54:06.344725 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:54:08.162563 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:54:08.162612 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:54:08.165064 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:54:08.165133 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:54:08.179716 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:54:08.179752 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:54:08.189790 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:54:08.189829 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:54:08.203645 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:54:08.203700 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:56:10.250449 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:56:10.250481 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"xfs"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f","csi.storage.k8s.io/pvc/name":"pvc-tq5wr","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","fsType":"xfs","networkAccessPolicy":"DenyAll","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f"} I0128 14:56:12.040466 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:56:12.040517 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:56:12.041038 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount with mount options([nouuid]) I0128 14:56:12.041072 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 14:56:12.051938 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=xfs\n" I0128 14:56:12.051969 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 14:56:12.063700 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in xfs format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount I0128 14:56:12.063743 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount) E0128 14:56:12.077991 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 14:56:12.078045 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t xfs -o nouuid,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a235c662-d7d2-4bc9-a9d2-3adc1106df7f/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 14:57:16.033153 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 14:57:16.033181 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-2a6e434b-866e-4971-a669-6b64223306b2/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-2a6e434b-866e-4971-a669-6b64223306b2","csi.storage.k8s.io/pvc/name":"pvc-pnkwg","csi.storage.k8s.io/pvc/namespace":"azuredisk-1598","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-2a6e434b-866e-4971-a669-6b64223306b2"} I0128 14:57:16.865155 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 14:57:16.865207 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 14:57:16.865719 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-2a6e434b-866e-4971-a669-6b64223306b2/globalmount with mount options([]) I0128 14:57:16.865743 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) ... skipping 180 lines ... I0128 14:59:51.893009 1 utils.go:84] GRPC response: {} I0128 14:59:51.911275 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 14:59:51.911306 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-81b06a61-b121-48d7-9355-3b86f4171a35","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-81b06a61-b121-48d7-9355-3b86f4171a35"} I0128 14:59:51.911386 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 I0128 14:59:51.911412 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-81b06a61-b121-48d7-9355-3b86f4171a35" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 14:59:51.911430 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 I0128 14:59:51.912893 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 I0128 14:59:51.912913 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-81b06a61-b121-48d7-9355-3b86f4171a35" I0128 14:59:51.913022 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-81b06a61-b121-48d7-9355-3b86f4171a35 successfully I0128 14:59:51.913042 1 utils.go:84] GRPC response: {} I0128 15:01:02.311737 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 15:01:02.311790 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-3d9450a0-a354-44cc-9143-1e628a4374e5","csi.storage.k8s.io/pvc/name":"pvc-jcdtv","csi.storage.k8s.io/pvc/namespace":"azuredisk-8582","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-3d9450a0-a354-44cc-9143-1e628a4374e5"} I0128 15:01:04.143327 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 562 lines ... I0128 15:22:42.733515 1 utils.go:84] GRPC response: {} I0128 15:22:42.768048 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 15:22:42.768076 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76"} I0128 15:22:42.768161 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 I0128 15:22:42.768191 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 15:22:42.768213 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 I0128 15:22:42.769672 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 I0128 15:22:42.769693 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76" I0128 15:22:42.769794 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 successfully I0128 15:22:42.769809 1 utils.go:84] GRPC response: {} I0128 15:23:53.166235 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 15:23:53.166289 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-6db6d396-e290-4392-91a4-4ccca47155e7/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-6db6d396-e290-4392-91a4-4ccca47155e7","csi.storage.k8s.io/pvc/name":"pvc-p5l4t","csi.storage.k8s.io/pvc/namespace":"azuredisk-1092","device-setting/device/queue_depth":"17","device-setting/queue/max_sectors_kb":"211","device-setting/queue/nr_requests":"44","device-setting/queue/read_ahead_kb":"256","device-setting/queue/rotational":"0","device-setting/queue/scheduler":"none","device-setting/queue/wbt_lat_usec":"0","perfProfile":"advanced","requestedsizegib":"10","skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-6db6d396-e290-4392-91a4-4ccca47155e7"} I0128 15:23:54.942578 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 100 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 14:05:21.534984 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-15a7100a0e7ccd86606f3f693bffc000489534a7 e2e-test I0128 14:05:21.535673 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 14:05:21.576869 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 14:05:21.576895 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 14:05:21.576905 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 14:05:21.577125 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 14:05:21.577998 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 14:05:21.578057 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 14:05:21.578144 1 azure.go:776] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 68 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 14:05:24.359259 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-15a7100a0e7ccd86606f3f693bffc000489534a7 e2e-test I0128 14:05:24.359856 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 14:05:24.391551 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 14:05:24.391567 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 14:05:24.391574 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 14:05:24.391598 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 14:05:24.392367 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 14:05:24.392400 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 14:05:24.392436 1 azure.go:776] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000 ... skipping 243 lines ... I0128 15:22:42.782307 1 utils.go:84] GRPC response: {} I0128 15:22:42.817454 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 15:22:42.817478 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-07b723c6-491f-4faa-96d7-28d557aded76"} I0128 15:22:42.817553 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 I0128 15:22:42.817577 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 15:22:42.817589 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 I0128 15:22:42.818761 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 I0128 15:22:42.818782 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76" I0128 15:22:42.818865 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-07b723c6-491f-4faa-96d7-28d557aded76 successfully I0128 15:22:42.818883 1 utils.go:84] GRPC response: {} I0128 15:25:25.462407 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 15:25:25.462695 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674914729268-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-zohsaios/providers/Microsoft.Compute/disks/pvc-9ba68df5-29fc-4ef0-9746-2912f7711bd9"} I0128 15:25:27.114367 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 594 lines ... cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-zohsaios",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="300"} 42 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-zohsaios",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="600"} 42 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-zohsaios",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="1200"} 42 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-zohsaios",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="+Inf"} 42 cloudprovider_azure_op_duration_seconds_sum{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-zohsaios",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 705.2798134779999 cloudprovider_azure_op_duration_seconds_count{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="kubetest-zohsaios",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 42 # HELP cloudprovider_azure_op_failure_count [ALPHA] Number of failed Azure service operations # TYPE cloudprovider_azure_op_failure_count counter cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_delete_volume",resource_group="kubetest-zohsaios",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 1 # HELP disabled_metric_total [ALPHA] The count of disabled metrics. # TYPE disabled_metric_total counter disabled_metric_total 0 # HELP go_cgo_go_to_c_calls_calls_total Count of calls made from Go to C by the current process. ... skipping 67 lines ... # HELP go_gc_heap_objects_objects Number of objects, live or unswept, occupying heap memory. # TYPE go_gc_heap_objects_objects gauge go_gc_heap_objects_objects 31522 # HELP go_gc_heap_tiny_allocs_objects_total Count of small allocations that are packed together into blocks. These allocations are counted separately from other allocations because each individual allocation is not tracked by the runtime, only their block. Each block is already accounted for in allocs-by-size and frees-by-size. # TYPE go_gc_heap_tiny_allocs_objects_total counter go_gc_heap_tiny_allocs_objects_total 45681 # HELP go_gc_limiter_last_enabled_gc_cycle GC cycle the last time the GC CPU limiter was enabled. This metric is useful for diagnosing the root cause of an out-of-memory error, because the limiter trades memory for CPU time when the GC's CPU time gets too high. This is most likely to occur with use of SetMemoryLimit. The first GC cycle is cycle 1, so a value of 0 indicates that it was never enabled. # TYPE go_gc_limiter_last_enabled_gc_cycle gauge go_gc_limiter_last_enabled_gc_cycle 0 # HELP go_gc_pauses_seconds Distribution individual GC-related stop-the-world pause latencies. # TYPE go_gc_pauses_seconds histogram go_gc_pauses_seconds_bucket{le="9.999999999999999e-10"} 0 go_gc_pauses_seconds_bucket{le="9.999999999999999e-09"} 0 ... skipping 259 lines ... # HELP go_gc_heap_objects_objects Number of objects, live or unswept, occupying heap memory. # TYPE go_gc_heap_objects_objects gauge go_gc_heap_objects_objects 16865 # HELP go_gc_heap_tiny_allocs_objects_total Count of small allocations that are packed together into blocks. These allocations are counted separately from other allocations because each individual allocation is not tracked by the runtime, only their block. Each block is already accounted for in allocs-by-size and frees-by-size. # TYPE go_gc_heap_tiny_allocs_objects_total counter go_gc_heap_tiny_allocs_objects_total 4108 # HELP go_gc_limiter_last_enabled_gc_cycle GC cycle the last time the GC CPU limiter was enabled. This metric is useful for diagnosing the root cause of an out-of-memory error, because the limiter trades memory for CPU time when the GC's CPU time gets too high. This is most likely to occur with use of SetMemoryLimit. The first GC cycle is cycle 1, so a value of 0 indicates that it was never enabled. # TYPE go_gc_limiter_last_enabled_gc_cycle gauge go_gc_limiter_last_enabled_gc_cycle 0 # HELP go_gc_pauses_seconds Distribution individual GC-related stop-the-world pause latencies. # TYPE go_gc_pauses_seconds histogram go_gc_pauses_seconds_bucket{le="9.999999999999999e-10"} 0 go_gc_pauses_seconds_bucket{le="9.999999999999999e-09"} 0 ... skipping 272 lines ... [AfterSuite] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/suite_test.go:165[0m [38;5;243m------------------------------[0m [38;5;9m[1mSummarizing 2 Failures:[0m [38;5;9m[FAIL][0m [0mDynamic Provisioning [38;5;243m[multi-az] [0m[38;5;9m[1m[It] should clone a volume from an existing volume and read from it [disk.csi.azure.com][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [38;5;9m[FAIL][0m [0mDynamic Provisioning [38;5;243m[multi-az] [0m[38;5;9m[1m[It] should clone a volume of larger size than the source volume and make sure the filesystem is appropriately adjusted [disk.csi.azure.com][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [38;5;9m[1mRan 26 of 66 Specs in 4907.112 seconds[0m [38;5;9m[1mFAIL![0m -- [38;5;10m[1m24 Passed[0m | [38;5;9m[1m2 Failed[0m | [38;5;11m[1m0 Pending[0m | [38;5;14m[1m40 Skipped[0m [38;5;228mYou're using deprecated Ginkgo functionality:[0m [38;5;228m=============================================[0m [38;5;11mSupport for custom reporters has been removed in V2. Please read the documentation linked to below for Ginkgo's new behavior and for a migration path:[0m [1mLearn more at:[0m [38;5;14m[4mhttps://onsi.github.io/ginkgo/MIGRATING_TO_V2#removed-custom-reporters[0m [38;5;243mTo silence deprecations that can be silenced set the following environment variable:[0m [38;5;243mACK_GINKGO_DEPRECATIONS=2.4.0[0m --- FAIL: TestE2E (4907.11s) FAIL FAIL sigs.k8s.io/azuredisk-csi-driver/test/e2e 4907.206s FAIL make: *** [Makefile:261: e2e-test] Error 1 2023/01/28 15:26:49 process.go:155: Step 'make e2e-test' finished in 1h23m26.497643395s 2023/01/28 15:26:49 aksengine_helpers.go:425: downloading /root/tmp1936064140/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh 2023/01/28 15:26:49 util.go:70: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh 2023/01/28 15:26:49 process.go:153: Running: chmod +x /root/tmp1936064140/log-dump.sh 2023/01/28 15:26:49 process.go:155: Step 'chmod +x /root/tmp1936064140/log-dump.sh' finished in 3.172201ms 2023/01/28 15:26:49 aksengine_helpers.go:425: downloading /root/tmp1936064140/log-dump-daemonset.yaml from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump-daemonset.yaml ... skipping 69 lines ... ssh key file /root/.ssh/id_rsa does not exist. Exiting. 2023/01/28 15:27:26 process.go:155: Step 'bash -c /root/tmp1936064140/win-ci-logs-collector.sh kubetest-zohsaios.westus2.cloudapp.azure.com /root/tmp1936064140 /root/.ssh/id_rsa' finished in 4.486317ms 2023/01/28 15:27:26 aksengine.go:1141: Deleting resource group: kubetest-zohsaios. 2023/01/28 15:34:32 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml. 2023/01/28 15:34:32 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}" 2023/01/28 15:34:32 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 265.053994ms 2023/01/28 15:34:32 main.go:328: Something went wrong: encountered 1 errors: [error during make e2e-test: exit status 2] + EXIT_VALUE=1 + set +o xtrace Cleaning up after docker in docker. ================================================================================ Cleaning up after docker Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die. ... skipping 3 lines ...