kubernetes-sigs/azuredisk-csi-driver PR #1704 pull-azuredisk-csi-driver-e2e-capz #1619364323593818112
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| PR | andyzhangx: fix: switch base image to fix CVEs |
| Result | FAILURE |
| Tests | 0 failed / 0 succeeded |
| Started | |
| Elapsed | 1h43m |
| Revision | a093a52191d3e3d9e4045b29bef24ef84b1ddc4f |
| Refs |
1704 |
... skipping 786 lines ... certificate.cert-manager.io "selfsigned-cert" deleted # Create secret for AzureClusterIdentity ./hack/create-identity-secret.sh make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure' make[2]: Nothing to be done for 'kubectl'. make[2]: Leaving directory '/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure' Error from server (NotFound): secrets "cluster-identity-secret" not found secret/cluster-identity-secret created secret/cluster-identity-secret labeled # Create customized cloud provider configs ./hack/create-custom-cloud-provider-config.sh make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure' make[2]: Nothing to be done for 'kubectl'. ... skipping 127 lines ... # Wait for the kubeconfig to become available. timeout --foreground 300 bash -c "while ! /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/hack/tools/bin/kubectl-v1.22.4 get secrets | grep capz-v4slna-kubeconfig; do sleep 1; done" capz-v4slna-kubeconfig cluster.x-k8s.io/secret 1 1s # Get kubeconfig and store it locally. /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/hack/tools/bin/kubectl-v1.22.4 get secrets capz-v4slna-kubeconfig -o json | jq -r .data.value | base64 --decode > ./kubeconfig timeout --foreground 600 bash -c "while ! /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/hack/tools/bin/kubectl-v1.22.4 --kubeconfig=./kubeconfig get nodes | grep control-plane; do sleep 1; done" error: the server doesn't have a resource type "nodes" capz-v4slna-control-plane-jxnw2 NotReady control-plane 3s v1.24.6 run "/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/hack/tools/bin/kubectl-v1.22.4 --kubeconfig=./kubeconfig ..." to work with the new target cluster make[1]: Leaving directory '/home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure' pod/kube-apiserver-capz-v4slna-control-plane-jxnw2 condition met namespace/calico-system created NAME DATA AGE ... skipping 78 lines ... 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 11345 100 11345 0 0 131k 0 --:--:-- --:--:-- --:--:-- 131k Downloading https://get.helm.sh/helm-v3.11.0-linux-amd64.tar.gz Verifying checksum... Done. Preparing to install helm into /usr/local/bin helm installed into /usr/local/bin/helm docker pull capzci.azurecr.io/azuredisk-csi:v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 || make container-all push-manifest Error response from daemon: manifest for capzci.azurecr.io/azuredisk-csi:v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 not found: manifest unknown: manifest tagged by "v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391" is not found make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver' CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=8635ef7cb96ec669bd2a099af3b1437a19530391 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2023-01-28T16:15:58Z -extldflags "-static"" -mod vendor -o _output/amd64/azurediskplugin.exe ./pkg/azurediskplugin docker buildx rm container-builder || true ERROR: no builder "container-builder" found docker buildx create --use --name=container-builder container-builder # enable qemu for arm64 build # https://github.com/docker/buildx/issues/464#issuecomment-741507760 docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64 Unable to find image 'tonistiigi/binfmt:latest' locally ... skipping 1094 lines ... type: string type: object oneOf: - required: ["persistentVolumeClaimName"] - required: ["volumeSnapshotContentName"] volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 60 lines ... type: string volumeSnapshotContentName: description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable. type: string type: object volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 254 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 108 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 865 lines ... image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.6.0" args: - "-csi-address=$(ADDRESS)" - "-v=2" - "-leader-election" - "--leader-election-namespace=kube-system" - '-handle-volume-inuse-error=false' - '-feature-gates=RecoverVolumeExpansionFailure=true' - "-timeout=240s" env: - name: ADDRESS value: /csi/csi.sock volumeMounts: ... skipping 217 lines ... Jan 28 16:23:29.184: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-vxkf2] to have phase Bound Jan 28 16:23:29.286: INFO: PersistentVolumeClaim pvc-vxkf2 found and phase=Bound (102.730492ms) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:23:29.286[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:23:29.389[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:23:29.493[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:23:29.493[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/28/23 16:23:29.598[0m Jan 28 16:23:29.598: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-wnszt" in namespace "azuredisk-8081" to be "Error status code" Jan 28 16:23:29.704: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 106.102941ms Jan 28 16:23:31.808: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.210292966s Jan 28 16:23:33.810: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 4.212151378s Jan 28 16:23:35.809: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 6.211691674s Jan 28 16:23:37.808: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 8.21063619s Jan 28 16:23:39.808: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 10.210465601s Jan 28 16:23:41.809: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 12.210777201s Jan 28 16:23:43.807: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 14.209410937s Jan 28 16:23:45.815: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 16.21741571s Jan 28 16:23:47.816: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 18.21845227s Jan 28 16:23:49.815: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Failed", Reason="", readiness=false. Elapsed: 20.217432131s [1mSTEP:[0m Saw pod failure [38;5;243m01/28/23 16:23:49.815[0m Jan 28 16:23:49.816: INFO: Pod "azuredisk-volume-tester-wnszt" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/28/23 16:23:49.816[0m Jan 28 16:23:49.939: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-wnszt" Jan 28 16:23:50.045: INFO: Pod azuredisk-volume-tester-wnszt has the following logs: /bin/sh: can't create /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-wnszt in namespace azuredisk-8081 [38;5;243m01/28/23 16:23:50.045[0m Jan 28 16:23:50.167: INFO: deleting PVC "azuredisk-8081"/"pvc-vxkf2" ... skipping 60 lines ... Jan 28 16:23:29.184: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-vxkf2] to have phase Bound Jan 28 16:23:29.286: INFO: PersistentVolumeClaim pvc-vxkf2 found and phase=Bound (102.730492ms) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:23:29.286[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:23:29.389[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:23:29.493[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:23:29.493[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/28/23 16:23:29.598[0m Jan 28 16:23:29.598: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-wnszt" in namespace "azuredisk-8081" to be "Error status code" Jan 28 16:23:29.704: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 106.102941ms Jan 28 16:23:31.808: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.210292966s Jan 28 16:23:33.810: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 4.212151378s Jan 28 16:23:35.809: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 6.211691674s Jan 28 16:23:37.808: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 8.21063619s Jan 28 16:23:39.808: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 10.210465601s Jan 28 16:23:41.809: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 12.210777201s Jan 28 16:23:43.807: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 14.209410937s Jan 28 16:23:45.815: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 16.21741571s Jan 28 16:23:47.816: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Pending", Reason="", readiness=false. Elapsed: 18.21845227s Jan 28 16:23:49.815: INFO: Pod "azuredisk-volume-tester-wnszt": Phase="Failed", Reason="", readiness=false. Elapsed: 20.217432131s [1mSTEP:[0m Saw pod failure [38;5;243m01/28/23 16:23:49.815[0m Jan 28 16:23:49.816: INFO: Pod "azuredisk-volume-tester-wnszt" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/28/23 16:23:49.816[0m Jan 28 16:23:49.939: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-wnszt" Jan 28 16:23:50.045: INFO: Pod azuredisk-volume-tester-wnszt has the following logs: /bin/sh: can't create /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-wnszt in namespace azuredisk-8081 [38;5;243m01/28/23 16:23:50.045[0m Jan 28 16:23:50.167: INFO: deleting PVC "azuredisk-8081"/"pvc-vxkf2" ... skipping 118 lines ... [1mSTEP:[0m Successfully provisioned a shared disk volume: "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/single-shared-disk" [38;5;243m01/28/23 16:24:28.594[0m [1mSTEP:[0m Destroying namespace "azuredisk-4728" for this suite. [38;5;243m01/28/23 16:24:29.346[0m [38;5;243m<< End Captured GinkgoWriter Output[0m [38;5;243m------------------------------[0m [0mPre-Provisioned [38;5;243m[single-az][0m [1mshould fail when maxShares is invalid [disk.csi.azure.com][windows][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:166[0m [1mSTEP:[0m Creating a kubernetes client [38;5;243m01/28/23 16:24:29.453[0m Jan 28 16:24:29.453: INFO: >>> kubeConfig: /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/kubeconfig [1mSTEP:[0m Building a namespace api object, basename azuredisk [38;5;243m01/28/23 16:24:29.454[0m [1mSTEP:[0m Waiting for a default service account to be provisioned in namespace [38;5;243m01/28/23 16:24:29.77[0m [1mSTEP:[0m Waiting for kube-root-ca.crt to be provisioned in namespace [38;5;243m01/28/23 16:24:29.975[0m ... skipping 2 lines ... [38;5;243m------------------------------[0m [38;5;10m• [0.834 seconds][0m Pre-Provisioned [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:38[0m [single-az] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:71[0m should fail when maxShares is invalid [disk.csi.azure.com][windows] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:166[0m [38;5;243mBegin Captured GinkgoWriter Output >>[0m [1mSTEP:[0m Creating a kubernetes client [38;5;243m01/28/23 16:24:29.453[0m Jan 28 16:24:29.453: INFO: >>> kubeConfig: /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/kubeconfig [1mSTEP:[0m Building a namespace api object, basename azuredisk [38;5;243m01/28/23 16:24:29.454[0m ... skipping 209 lines ... Jan 28 16:26:20.650: INFO: PersistentVolumeClaim pvc-br5nl found and phase=Bound (103.105846ms) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:26:20.65[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:26:20.754[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:26:20.858[0m [1mSTEP:[0m attaching disk to node#0 [38;5;243m01/28/23 16:26:21.025[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:26:34.142[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:26:34.247[0m Jan 28 16:26:34.247: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-bbxql" in namespace "azuredisk-563" to be "Succeeded or Failed" Jan 28 16:26:34.350: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 103.237373ms Jan 28 16:26:36.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207723126s Jan 28 16:26:38.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207748359s Jan 28 16:26:40.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20767162s Jan 28 16:26:42.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 8.208114681s Jan 28 16:26:44.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 10.207665701s ... skipping 7 lines ... Jan 28 16:27:00.454: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 26.206726868s Jan 28 16:27:02.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 28.207924177s Jan 28 16:27:04.461: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 30.213768078s Jan 28 16:27:06.461: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 32.213891679s Jan 28 16:27:08.460: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Succeeded", Reason="", readiness=false. Elapsed: 34.213153509s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:27:08.46[0m Jan 28 16:27:08.460: INFO: Pod "azuredisk-volume-tester-bbxql" satisfied condition "Succeeded or Failed" Jan 28 16:27:08.460: INFO: deleting Pod "azuredisk-563"/"azuredisk-volume-tester-bbxql" Jan 28 16:27:08.574: INFO: Pod azuredisk-volume-tester-bbxql has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-bbxql in namespace azuredisk-563 [38;5;243m01/28/23 16:27:08.575[0m Jan 28 16:27:09.044: INFO: deleting PVC "azuredisk-563"/"pvc-br5nl" Jan 28 16:27:09.044: INFO: Deleting PersistentVolumeClaim "pvc-br5nl" ... skipping 35 lines ... Jan 28 16:26:20.650: INFO: PersistentVolumeClaim pvc-br5nl found and phase=Bound (103.105846ms) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:26:20.65[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:26:20.754[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:26:20.858[0m [1mSTEP:[0m attaching disk to node#0 [38;5;243m01/28/23 16:26:21.025[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:26:34.142[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:26:34.247[0m Jan 28 16:26:34.247: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-bbxql" in namespace "azuredisk-563" to be "Succeeded or Failed" Jan 28 16:26:34.350: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 103.237373ms Jan 28 16:26:36.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207723126s Jan 28 16:26:38.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207748359s Jan 28 16:26:40.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20767162s Jan 28 16:26:42.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 8.208114681s Jan 28 16:26:44.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 10.207665701s ... skipping 7 lines ... Jan 28 16:27:00.454: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 26.206726868s Jan 28 16:27:02.455: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 28.207924177s Jan 28 16:27:04.461: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 30.213768078s Jan 28 16:27:06.461: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Pending", Reason="", readiness=false. Elapsed: 32.213891679s Jan 28 16:27:08.460: INFO: Pod "azuredisk-volume-tester-bbxql": Phase="Succeeded", Reason="", readiness=false. Elapsed: 34.213153509s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:27:08.46[0m Jan 28 16:27:08.460: INFO: Pod "azuredisk-volume-tester-bbxql" satisfied condition "Succeeded or Failed" Jan 28 16:27:08.460: INFO: deleting Pod "azuredisk-563"/"azuredisk-volume-tester-bbxql" Jan 28 16:27:08.574: INFO: Pod azuredisk-volume-tester-bbxql has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-bbxql in namespace azuredisk-563 [38;5;243m01/28/23 16:27:08.575[0m Jan 28 16:27:09.044: INFO: deleting PVC "azuredisk-563"/"pvc-br5nl" Jan 28 16:27:09.044: INFO: Deleting PersistentVolumeClaim "pvc-br5nl" ... skipping 66 lines ... Jan 28 16:27:38.123: INFO: PersistentVolumeClaim pvc-lpd54 found and phase=Bound (4.312301257s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:27:38.123[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:27:38.228[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:27:38.332[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:27:38.332[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:27:38.332[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:27:38.437[0m Jan 28 16:27:38.437: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-t58hv" in namespace "azuredisk-4538" to be "Succeeded or Failed" Jan 28 16:27:38.542: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 105.631873ms Jan 28 16:27:40.646: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.208848157s Jan 28 16:27:42.647: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.210093817s Jan 28 16:27:44.646: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.209300716s Jan 28 16:27:46.648: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.211037524s Jan 28 16:27:48.647: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.209963315s Jan 28 16:27:50.646: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 12.209009036s Jan 28 16:27:52.647: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 14.209903987s Jan 28 16:27:54.652: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 16.215694649s Jan 28 16:27:56.652: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.214885686s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:27:56.652[0m Jan 28 16:27:56.652: INFO: Pod "azuredisk-volume-tester-t58hv" satisfied condition "Succeeded or Failed" Jan 28 16:27:56.652: INFO: deleting Pod "azuredisk-4538"/"azuredisk-volume-tester-t58hv" Jan 28 16:27:56.758: INFO: Pod azuredisk-volume-tester-t58hv has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-t58hv in namespace azuredisk-4538 [38;5;243m01/28/23 16:27:56.758[0m Jan 28 16:27:56.884: INFO: deleting PVC "azuredisk-4538"/"pvc-lpd54" Jan 28 16:27:56.884: INFO: Deleting PersistentVolumeClaim "pvc-lpd54" ... skipping 39 lines ... Jan 28 16:27:38.123: INFO: PersistentVolumeClaim pvc-lpd54 found and phase=Bound (4.312301257s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:27:38.123[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:27:38.228[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:27:38.332[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:27:38.332[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:27:38.332[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:27:38.437[0m Jan 28 16:27:38.437: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-t58hv" in namespace "azuredisk-4538" to be "Succeeded or Failed" Jan 28 16:27:38.542: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 105.631873ms Jan 28 16:27:40.646: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.208848157s Jan 28 16:27:42.647: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.210093817s Jan 28 16:27:44.646: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.209300716s Jan 28 16:27:46.648: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.211037524s Jan 28 16:27:48.647: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.209963315s Jan 28 16:27:50.646: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 12.209009036s Jan 28 16:27:52.647: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 14.209903987s Jan 28 16:27:54.652: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Pending", Reason="", readiness=false. Elapsed: 16.215694649s Jan 28 16:27:56.652: INFO: Pod "azuredisk-volume-tester-t58hv": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.214885686s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:27:56.652[0m Jan 28 16:27:56.652: INFO: Pod "azuredisk-volume-tester-t58hv" satisfied condition "Succeeded or Failed" Jan 28 16:27:56.652: INFO: deleting Pod "azuredisk-4538"/"azuredisk-volume-tester-t58hv" Jan 28 16:27:56.758: INFO: Pod azuredisk-volume-tester-t58hv has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-t58hv in namespace azuredisk-4538 [38;5;243m01/28/23 16:27:56.758[0m Jan 28 16:27:56.884: INFO: deleting PVC "azuredisk-4538"/"pvc-lpd54" Jan 28 16:27:56.884: INFO: Deleting PersistentVolumeClaim "pvc-lpd54" ... skipping 37 lines ... Jan 28 16:28:42.118: INFO: PersistentVolumeClaim pvc-cl8zr found but phase is Pending instead of Bound. Jan 28 16:28:44.222: INFO: PersistentVolumeClaim pvc-cl8zr found and phase=Bound (4.310848254s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:28:44.222[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:28:44.325[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:28:44.429[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:28:44.429[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m01/28/23 16:28:44.534[0m Jan 28 16:28:44.535: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-mdd45" in namespace "azuredisk-8266" to be "Succeeded or Failed" Jan 28 16:28:44.638: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 103.197643ms Jan 28 16:28:46.742: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207270894s Jan 28 16:28:48.742: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 4.20734374s Jan 28 16:28:50.743: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 6.208133851s Jan 28 16:28:52.742: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 8.207863111s Jan 28 16:28:54.743: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 10.208196008s Jan 28 16:28:56.743: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 12.207992983s Jan 28 16:28:58.741: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 14.206645711s Jan 28 16:29:00.748: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 16.213103744s Jan 28 16:29:02.748: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.213852504s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:29:02.748[0m Jan 28 16:29:02.749: INFO: Pod "azuredisk-volume-tester-mdd45" satisfied condition "Succeeded or Failed" Jan 28 16:29:02.749: INFO: deleting Pod "azuredisk-8266"/"azuredisk-volume-tester-mdd45" Jan 28 16:29:02.861: INFO: Pod azuredisk-volume-tester-mdd45 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-mdd45 in namespace azuredisk-8266 [38;5;243m01/28/23 16:29:02.861[0m Jan 28 16:29:02.976: INFO: deleting PVC "azuredisk-8266"/"pvc-cl8zr" Jan 28 16:29:02.976: INFO: Deleting PersistentVolumeClaim "pvc-cl8zr" ... skipping 38 lines ... Jan 28 16:28:42.118: INFO: PersistentVolumeClaim pvc-cl8zr found but phase is Pending instead of Bound. Jan 28 16:28:44.222: INFO: PersistentVolumeClaim pvc-cl8zr found and phase=Bound (4.310848254s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:28:44.222[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:28:44.325[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:28:44.429[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:28:44.429[0m [1mSTEP:[0m checking that the pods command exits with no error [38;5;243m01/28/23 16:28:44.534[0m Jan 28 16:28:44.535: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-mdd45" in namespace "azuredisk-8266" to be "Succeeded or Failed" Jan 28 16:28:44.638: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 103.197643ms Jan 28 16:28:46.742: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207270894s Jan 28 16:28:48.742: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 4.20734374s Jan 28 16:28:50.743: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 6.208133851s Jan 28 16:28:52.742: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 8.207863111s Jan 28 16:28:54.743: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 10.208196008s Jan 28 16:28:56.743: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 12.207992983s Jan 28 16:28:58.741: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 14.206645711s Jan 28 16:29:00.748: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Pending", Reason="", readiness=false. Elapsed: 16.213103744s Jan 28 16:29:02.748: INFO: Pod "azuredisk-volume-tester-mdd45": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.213852504s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:29:02.748[0m Jan 28 16:29:02.749: INFO: Pod "azuredisk-volume-tester-mdd45" satisfied condition "Succeeded or Failed" Jan 28 16:29:02.749: INFO: deleting Pod "azuredisk-8266"/"azuredisk-volume-tester-mdd45" Jan 28 16:29:02.861: INFO: Pod azuredisk-volume-tester-mdd45 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-mdd45 in namespace azuredisk-8266 [38;5;243m01/28/23 16:29:02.861[0m Jan 28 16:29:02.976: INFO: deleting PVC "azuredisk-8266"/"pvc-cl8zr" Jan 28 16:29:02.976: INFO: Deleting PersistentVolumeClaim "pvc-cl8zr" ... skipping 38 lines ... Jan 28 16:29:50.317: INFO: PersistentVolumeClaim pvc-8xgtl found and phase=Bound (4.323327915s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:29:50.317[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:29:50.427[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:29:50.53[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:29:50.531[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:29:50.531[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:29:50.636[0m Jan 28 16:29:50.636: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-2wq9t" in namespace "azuredisk-4376" to be "Succeeded or Failed" Jan 28 16:29:50.739: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 102.829521ms Jan 28 16:29:52.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207055271s Jan 28 16:29:54.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 4.206630816s Jan 28 16:29:56.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20716914s Jan 28 16:29:58.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 8.207252249s Jan 28 16:30:00.844: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 10.207749269s ... skipping 15 lines ... Jan 28 16:30:32.842: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 42.206106112s Jan 28 16:30:34.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 44.206922647s Jan 28 16:30:36.850: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 46.213909076s Jan 28 16:30:38.850: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 48.213993273s Jan 28 16:30:40.849: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Succeeded", Reason="", readiness=false. Elapsed: 50.2126023s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:30:40.849[0m Jan 28 16:30:40.849: INFO: Pod "azuredisk-volume-tester-2wq9t" satisfied condition "Succeeded or Failed" Jan 28 16:30:40.849: INFO: deleting Pod "azuredisk-4376"/"azuredisk-volume-tester-2wq9t" Jan 28 16:30:40.966: INFO: Pod azuredisk-volume-tester-2wq9t has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-2wq9t in namespace azuredisk-4376 [38;5;243m01/28/23 16:30:40.966[0m Jan 28 16:30:41.092: INFO: deleting PVC "azuredisk-4376"/"pvc-8xgtl" Jan 28 16:30:41.092: INFO: Deleting PersistentVolumeClaim "pvc-8xgtl" ... skipping 39 lines ... Jan 28 16:29:50.317: INFO: PersistentVolumeClaim pvc-8xgtl found and phase=Bound (4.323327915s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:29:50.317[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:29:50.427[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:29:50.53[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:29:50.531[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:29:50.531[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:29:50.636[0m Jan 28 16:29:50.636: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-2wq9t" in namespace "azuredisk-4376" to be "Succeeded or Failed" Jan 28 16:29:50.739: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 102.829521ms Jan 28 16:29:52.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 2.207055271s Jan 28 16:29:54.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 4.206630816s Jan 28 16:29:56.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20716914s Jan 28 16:29:58.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 8.207252249s Jan 28 16:30:00.844: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 10.207749269s ... skipping 15 lines ... Jan 28 16:30:32.842: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 42.206106112s Jan 28 16:30:34.843: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 44.206922647s Jan 28 16:30:36.850: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 46.213909076s Jan 28 16:30:38.850: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Pending", Reason="", readiness=false. Elapsed: 48.213993273s Jan 28 16:30:40.849: INFO: Pod "azuredisk-volume-tester-2wq9t": Phase="Succeeded", Reason="", readiness=false. Elapsed: 50.2126023s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:30:40.849[0m Jan 28 16:30:40.849: INFO: Pod "azuredisk-volume-tester-2wq9t" satisfied condition "Succeeded or Failed" Jan 28 16:30:40.849: INFO: deleting Pod "azuredisk-4376"/"azuredisk-volume-tester-2wq9t" Jan 28 16:30:40.966: INFO: Pod azuredisk-volume-tester-2wq9t has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-2wq9t in namespace azuredisk-4376 [38;5;243m01/28/23 16:30:40.966[0m Jan 28 16:30:41.092: INFO: deleting PVC "azuredisk-4376"/"pvc-8xgtl" Jan 28 16:30:41.092: INFO: Deleting PersistentVolumeClaim "pvc-8xgtl" ... skipping 40 lines ... [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:31:28.538[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:31:28.642[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:31:28.643[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:31:28.643[0m [1mSTEP:[0m checking that the pod has 'FailedMount' event [38;5;243m01/28/23 16:31:28.748[0m Jan 28 16:31:42.955: INFO: deleting Pod "azuredisk-7996"/"azuredisk-volume-tester-ltlb5" Jan 28 16:31:43.060: INFO: Error getting logs for pod azuredisk-volume-tester-ltlb5: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-ltlb5) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-ltlb5 in namespace azuredisk-7996 [38;5;243m01/28/23 16:31:43.061[0m Jan 28 16:31:43.167: INFO: deleting PVC "azuredisk-7996"/"pvc-rg46b" Jan 28 16:31:43.168: INFO: Deleting PersistentVolumeClaim "pvc-rg46b" [1mSTEP:[0m waiting for claim's PV "pvc-ed2992de-bd14-4d3d-a766-85404c0f727f" to be deleted [38;5;243m01/28/23 16:31:43.273[0m Jan 28 16:31:43.273: INFO: Waiting up to 10m0s for PersistentVolume pvc-ed2992de-bd14-4d3d-a766-85404c0f727f to get deleted Jan 28 16:31:43.377: INFO: PersistentVolume pvc-ed2992de-bd14-4d3d-a766-85404c0f727f found and phase=Bound (104.031299ms) ... skipping 59 lines ... [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:31:28.538[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:31:28.642[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:31:28.643[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:31:28.643[0m [1mSTEP:[0m checking that the pod has 'FailedMount' event [38;5;243m01/28/23 16:31:28.748[0m Jan 28 16:31:42.955: INFO: deleting Pod "azuredisk-7996"/"azuredisk-volume-tester-ltlb5" Jan 28 16:31:43.060: INFO: Error getting logs for pod azuredisk-volume-tester-ltlb5: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-ltlb5) [1mSTEP:[0m Deleting pod azuredisk-volume-tester-ltlb5 in namespace azuredisk-7996 [38;5;243m01/28/23 16:31:43.061[0m Jan 28 16:31:43.167: INFO: deleting PVC "azuredisk-7996"/"pvc-rg46b" Jan 28 16:31:43.168: INFO: Deleting PersistentVolumeClaim "pvc-rg46b" [1mSTEP:[0m waiting for claim's PV "pvc-ed2992de-bd14-4d3d-a766-85404c0f727f" to be deleted [38;5;243m01/28/23 16:31:43.273[0m Jan 28 16:31:43.273: INFO: Waiting up to 10m0s for PersistentVolume pvc-ed2992de-bd14-4d3d-a766-85404c0f727f to get deleted Jan 28 16:31:43.377: INFO: PersistentVolume pvc-ed2992de-bd14-4d3d-a766-85404c0f727f found and phase=Bound (104.031299ms) ... skipping 56 lines ... Jan 28 16:34:17.895: INFO: PersistentVolumeClaim pvc-vnkqr found and phase=Bound (4.312481829s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:34:17.896[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:34:18[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:34:18.104[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:34:18.104[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:34:18.104[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:34:18.21[0m Jan 28 16:34:18.210: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-jwrc5" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 28 16:34:18.315: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 104.582087ms Jan 28 16:34:20.422: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.211087048s Jan 28 16:34:22.421: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.210411079s Jan 28 16:34:24.419: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.208904839s Jan 28 16:34:26.420: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209266493s Jan 28 16:34:28.422: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.21107786s Jan 28 16:34:30.427: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 12.216508026s Jan 28 16:34:32.427: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Running", Reason="", readiness=false. Elapsed: 14.216874067s Jan 28 16:34:34.427: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 16.216831326s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:34:34.427[0m Jan 28 16:34:34.428: INFO: Pod "azuredisk-volume-tester-jwrc5" satisfied condition "Succeeded or Failed" Jan 28 16:34:34.428: INFO: deleting Pod "azuredisk-59"/"azuredisk-volume-tester-jwrc5" Jan 28 16:34:34.549: INFO: Pod azuredisk-volume-tester-jwrc5 has the following logs: e2e-test [1mSTEP:[0m Deleting pod azuredisk-volume-tester-jwrc5 in namespace azuredisk-59 [38;5;243m01/28/23 16:34:34.549[0m Jan 28 16:34:34.669: INFO: deleting PVC "azuredisk-59"/"pvc-vnkqr" Jan 28 16:34:34.669: INFO: Deleting PersistentVolumeClaim "pvc-vnkqr" ... skipping 39 lines ... Jan 28 16:34:17.895: INFO: PersistentVolumeClaim pvc-vnkqr found and phase=Bound (4.312481829s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:34:17.896[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:34:18[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:34:18.104[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:34:18.104[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:34:18.104[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:34:18.21[0m Jan 28 16:34:18.210: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-jwrc5" in namespace "azuredisk-59" to be "Succeeded or Failed" Jan 28 16:34:18.315: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 104.582087ms Jan 28 16:34:20.422: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.211087048s Jan 28 16:34:22.421: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.210411079s Jan 28 16:34:24.419: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.208904839s Jan 28 16:34:26.420: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209266493s Jan 28 16:34:28.422: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.21107786s Jan 28 16:34:30.427: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Pending", Reason="", readiness=false. Elapsed: 12.216508026s Jan 28 16:34:32.427: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Running", Reason="", readiness=false. Elapsed: 14.216874067s Jan 28 16:34:34.427: INFO: Pod "azuredisk-volume-tester-jwrc5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 16.216831326s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:34:34.427[0m Jan 28 16:34:34.428: INFO: Pod "azuredisk-volume-tester-jwrc5" satisfied condition "Succeeded or Failed" Jan 28 16:34:34.428: INFO: deleting Pod "azuredisk-59"/"azuredisk-volume-tester-jwrc5" Jan 28 16:34:34.549: INFO: Pod azuredisk-volume-tester-jwrc5 has the following logs: e2e-test [1mSTEP:[0m Deleting pod azuredisk-volume-tester-jwrc5 in namespace azuredisk-59 [38;5;243m01/28/23 16:34:34.549[0m Jan 28 16:34:34.669: INFO: deleting PVC "azuredisk-59"/"pvc-vnkqr" Jan 28 16:34:34.669: INFO: Deleting PersistentVolumeClaim "pvc-vnkqr" ... skipping 41 lines ... Jan 28 16:35:21.859: INFO: PersistentVolumeClaim pvc-vjwb5 found but phase is Pending instead of Bound. Jan 28 16:35:23.964: INFO: PersistentVolumeClaim pvc-vjwb5 found and phase=Bound (4.313822804s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:35:23.965[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:35:24.073[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:35:24.177[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:35:24.178[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:35:24.282[0m Jan 28 16:35:24.282: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7gqj6" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 28 16:35:24.388: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 105.213434ms Jan 28 16:35:26.494: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.211418276s Jan 28 16:35:28.496: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.213283927s Jan 28 16:35:30.491: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20885284s Jan 28 16:35:32.493: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.2106749s Jan 28 16:35:34.498: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Running", Reason="", readiness=true. Elapsed: 10.215516646s Jan 28 16:35:36.498: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Running", Reason="", readiness=false. Elapsed: 12.215911051s Jan 28 16:35:38.499: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 14.216748098s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:35:38.499[0m Jan 28 16:35:38.500: INFO: Pod "azuredisk-volume-tester-7gqj6" satisfied condition "Succeeded or Failed" Jan 28 16:35:38.500: INFO: deleting Pod "azuredisk-2546"/"azuredisk-volume-tester-7gqj6" Jan 28 16:35:38.607: INFO: Pod azuredisk-volume-tester-7gqj6 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7gqj6 in namespace azuredisk-2546 [38;5;243m01/28/23 16:35:38.607[0m Jan 28 16:35:38.724: INFO: deleting PVC "azuredisk-2546"/"pvc-vjwb5" Jan 28 16:35:38.724: INFO: Deleting PersistentVolumeClaim "pvc-vjwb5" ... skipping 39 lines ... Jan 28 16:35:21.859: INFO: PersistentVolumeClaim pvc-vjwb5 found but phase is Pending instead of Bound. Jan 28 16:35:23.964: INFO: PersistentVolumeClaim pvc-vjwb5 found and phase=Bound (4.313822804s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:35:23.965[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:35:24.073[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:35:24.177[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:35:24.178[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:35:24.282[0m Jan 28 16:35:24.282: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-7gqj6" in namespace "azuredisk-2546" to be "Succeeded or Failed" Jan 28 16:35:24.388: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 105.213434ms Jan 28 16:35:26.494: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.211418276s Jan 28 16:35:28.496: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.213283927s Jan 28 16:35:30.491: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20885284s Jan 28 16:35:32.493: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.2106749s Jan 28 16:35:34.498: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Running", Reason="", readiness=true. Elapsed: 10.215516646s Jan 28 16:35:36.498: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Running", Reason="", readiness=false. Elapsed: 12.215911051s Jan 28 16:35:38.499: INFO: Pod "azuredisk-volume-tester-7gqj6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 14.216748098s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:35:38.499[0m Jan 28 16:35:38.500: INFO: Pod "azuredisk-volume-tester-7gqj6" satisfied condition "Succeeded or Failed" Jan 28 16:35:38.500: INFO: deleting Pod "azuredisk-2546"/"azuredisk-volume-tester-7gqj6" Jan 28 16:35:38.607: INFO: Pod azuredisk-volume-tester-7gqj6 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-7gqj6 in namespace azuredisk-2546 [38;5;243m01/28/23 16:35:38.607[0m Jan 28 16:35:38.724: INFO: deleting PVC "azuredisk-2546"/"pvc-vjwb5" Jan 28 16:35:38.724: INFO: Deleting PersistentVolumeClaim "pvc-vjwb5" ... skipping 52 lines ... Jan 28 16:37:40.132: INFO: PersistentVolumeClaim pvc-gj9dp found but phase is Pending instead of Bound. Jan 28 16:37:42.236: INFO: PersistentVolumeClaim pvc-gj9dp found and phase=Bound (4.310109128s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:37:42.237[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:37:42.339[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:37:42.441[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:37:42.441[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:37:42.545[0m Jan 28 16:37:42.545: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-4xb9x" in namespace "azuredisk-1598" to be "Succeeded or Failed" Jan 28 16:37:42.653: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 107.311841ms Jan 28 16:37:44.756: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 2.21048014s Jan 28 16:37:46.757: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 4.211577716s Jan 28 16:37:48.756: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 6.210578698s Jan 28 16:37:50.754: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209238276s Jan 28 16:37:52.756: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 10.2108992s ... skipping 2 lines ... Jan 28 16:37:58.758: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 16.212396019s Jan 28 16:38:00.757: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 18.211689273s Jan 28 16:38:02.763: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Running", Reason="", readiness=true. Elapsed: 20.217605037s Jan 28 16:38:04.764: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Running", Reason="", readiness=false. Elapsed: 22.218930025s Jan 28 16:38:06.766: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.220629982s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:38:06.766[0m Jan 28 16:38:06.766: INFO: Pod "azuredisk-volume-tester-4xb9x" satisfied condition "Succeeded or Failed" Jan 28 16:38:06.766: INFO: deleting Pod "azuredisk-1598"/"azuredisk-volume-tester-4xb9x" Jan 28 16:38:06.885: INFO: Pod azuredisk-volume-tester-4xb9x has the following logs: hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-4xb9x in namespace azuredisk-1598 [38;5;243m01/28/23 16:38:06.885[0m Jan 28 16:38:07.004: INFO: deleting PVC "azuredisk-1598"/"pvc-gj9dp" ... skipping 72 lines ... Jan 28 16:37:40.132: INFO: PersistentVolumeClaim pvc-gj9dp found but phase is Pending instead of Bound. Jan 28 16:37:42.236: INFO: PersistentVolumeClaim pvc-gj9dp found and phase=Bound (4.310109128s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:37:42.237[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:37:42.339[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:37:42.441[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:37:42.441[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:37:42.545[0m Jan 28 16:37:42.545: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-4xb9x" in namespace "azuredisk-1598" to be "Succeeded or Failed" Jan 28 16:37:42.653: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 107.311841ms Jan 28 16:37:44.756: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 2.21048014s Jan 28 16:37:46.757: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 4.211577716s Jan 28 16:37:48.756: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 6.210578698s Jan 28 16:37:50.754: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209238276s Jan 28 16:37:52.756: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 10.2108992s ... skipping 2 lines ... Jan 28 16:37:58.758: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 16.212396019s Jan 28 16:38:00.757: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Pending", Reason="", readiness=false. Elapsed: 18.211689273s Jan 28 16:38:02.763: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Running", Reason="", readiness=true. Elapsed: 20.217605037s Jan 28 16:38:04.764: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Running", Reason="", readiness=false. Elapsed: 22.218930025s Jan 28 16:38:06.766: INFO: Pod "azuredisk-volume-tester-4xb9x": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.220629982s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:38:06.766[0m Jan 28 16:38:06.766: INFO: Pod "azuredisk-volume-tester-4xb9x" satisfied condition "Succeeded or Failed" Jan 28 16:38:06.766: INFO: deleting Pod "azuredisk-1598"/"azuredisk-volume-tester-4xb9x" Jan 28 16:38:06.885: INFO: Pod azuredisk-volume-tester-4xb9x has the following logs: hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-4xb9x in namespace azuredisk-1598 [38;5;243m01/28/23 16:38:06.885[0m Jan 28 16:38:07.004: INFO: deleting PVC "azuredisk-1598"/"pvc-gj9dp" ... skipping 54 lines ... Jan 28 16:41:56.555: INFO: PersistentVolumeClaim pvc-25bdm found and phase=Bound (4.311260104s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:41:56.555[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:41:56.657[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:41:56.76[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:41:56.76[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:41:56.76[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/28/23 16:41:56.865[0m Jan 28 16:41:56.865: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-6tx6v" in namespace "azuredisk-3410" to be "Error status code" Jan 28 16:41:56.970: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 104.934663ms Jan 28 16:41:59.074: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 2.208557588s Jan 28 16:42:01.075: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 4.209825889s Jan 28 16:42:03.074: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 6.208291904s Jan 28 16:42:05.075: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209338685s Jan 28 16:42:07.075: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 10.209678057s Jan 28 16:42:09.074: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 12.208991161s Jan 28 16:42:11.082: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 14.216997028s Jan 28 16:42:13.082: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 16.216841155s Jan 28 16:42:15.082: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Failed", Reason="", readiness=false. Elapsed: 18.2168912s [1mSTEP:[0m Saw pod failure [38;5;243m01/28/23 16:42:15.082[0m Jan 28 16:42:15.082: INFO: Pod "azuredisk-volume-tester-6tx6v" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/28/23 16:42:15.082[0m Jan 28 16:42:15.206: INFO: deleting Pod "azuredisk-3410"/"azuredisk-volume-tester-6tx6v" Jan 28 16:42:15.310: INFO: Pod azuredisk-volume-tester-6tx6v has the following logs: touch: /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-6tx6v in namespace azuredisk-3410 [38;5;243m01/28/23 16:42:15.311[0m Jan 28 16:42:15.430: INFO: deleting PVC "azuredisk-3410"/"pvc-25bdm" ... skipping 40 lines ... Jan 28 16:41:56.555: INFO: PersistentVolumeClaim pvc-25bdm found and phase=Bound (4.311260104s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:41:56.555[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:41:56.657[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:41:56.76[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:41:56.76[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:41:56.76[0m [1mSTEP:[0m checking that the pod's command exits with an error [38;5;243m01/28/23 16:41:56.865[0m Jan 28 16:41:56.865: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-6tx6v" in namespace "azuredisk-3410" to be "Error status code" Jan 28 16:41:56.970: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 104.934663ms Jan 28 16:41:59.074: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 2.208557588s Jan 28 16:42:01.075: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 4.209825889s Jan 28 16:42:03.074: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 6.208291904s Jan 28 16:42:05.075: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209338685s Jan 28 16:42:07.075: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 10.209678057s Jan 28 16:42:09.074: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 12.208991161s Jan 28 16:42:11.082: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 14.216997028s Jan 28 16:42:13.082: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Pending", Reason="", readiness=false. Elapsed: 16.216841155s Jan 28 16:42:15.082: INFO: Pod "azuredisk-volume-tester-6tx6v": Phase="Failed", Reason="", readiness=false. Elapsed: 18.2168912s [1mSTEP:[0m Saw pod failure [38;5;243m01/28/23 16:42:15.082[0m Jan 28 16:42:15.082: INFO: Pod "azuredisk-volume-tester-6tx6v" satisfied condition "Error status code" [1mSTEP:[0m checking that pod logs contain expected message [38;5;243m01/28/23 16:42:15.082[0m Jan 28 16:42:15.206: INFO: deleting Pod "azuredisk-3410"/"azuredisk-volume-tester-6tx6v" Jan 28 16:42:15.310: INFO: Pod azuredisk-volume-tester-6tx6v has the following logs: touch: /mnt/test-1/data: Read-only file system [1mSTEP:[0m Deleting pod azuredisk-volume-tester-6tx6v in namespace azuredisk-3410 [38;5;243m01/28/23 16:42:15.311[0m Jan 28 16:42:15.430: INFO: deleting PVC "azuredisk-3410"/"pvc-25bdm" ... skipping 725 lines ... Jan 28 16:50:59.052: INFO: PersistentVolumeClaim pvc-cwjgv found and phase=Bound (4.307952s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:50:59.052[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:50:59.154[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:50:59.271[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:50:59.271[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:50:59.271[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:50:59.378[0m Jan 28 16:50:59.378: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-wn49t" in namespace "azuredisk-3612" to be "Succeeded or Failed" Jan 28 16:50:59.480: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 102.01235ms Jan 28 16:51:01.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205955768s Jan 28 16:51:03.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 4.206587899s Jan 28 16:51:05.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206216222s Jan 28 16:51:07.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206700851s Jan 28 16:51:09.582: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 10.204658354s Jan 28 16:51:11.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 12.206354177s Jan 28 16:51:13.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205772929s Jan 28 16:51:15.591: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 16.212727937s Jan 28 16:51:17.590: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.211854133s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:51:17.59[0m Jan 28 16:51:17.590: INFO: Pod "azuredisk-volume-tester-wn49t" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/28/23 16:51:17.59[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/28/23 16:51:22.591[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 16:51:22.795[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 16:51:22.795[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m01/28/23 16:51:22.905[0m Jan 28 16:51:22.905: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-5pfg4] to have phase Bound ... skipping 2 lines ... Jan 28 16:51:27.213: INFO: PersistentVolumeClaim pvc-5pfg4 found and phase=Bound (4.308476864s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:51:27.213[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:51:27.317[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:51:27.419[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:51:27.419[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/28/23 16:51:27.419[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:51:27.524[0m Jan 28 16:51:27.524: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-766w8" in namespace "azuredisk-3612" to be "Succeeded or Failed" Jan 28 16:51:27.625: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 101.616058ms Jan 28 16:51:29.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204044208s Jan 28 16:51:31.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204520474s Jan 28 16:51:33.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 6.204242889s Jan 28 16:51:35.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 8.204511453s Jan 28 16:51:37.729: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205510069s ... skipping 2 lines ... Jan 28 16:51:43.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 16.204442385s Jan 28 16:51:45.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 18.204711405s Jan 28 16:51:47.737: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 20.213376437s Jan 28 16:51:49.735: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 22.211212966s Jan 28 16:51:51.735: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.211455012s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:51:51.735[0m Jan 28 16:51:51.735: INFO: Pod "azuredisk-volume-tester-766w8" satisfied condition "Succeeded or Failed" Jan 28 16:51:51.735: INFO: deleting Pod "azuredisk-3612"/"azuredisk-volume-tester-766w8" Jan 28 16:51:51.856: INFO: Pod azuredisk-volume-tester-766w8 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-766w8 in namespace azuredisk-3612 [38;5;243m01/28/23 16:51:51.856[0m Jan 28 16:51:51.984: INFO: deleting PVC "azuredisk-3612"/"pvc-5pfg4" Jan 28 16:51:51.984: INFO: Deleting PersistentVolumeClaim "pvc-5pfg4" ... skipping 51 lines ... Jan 28 16:50:59.052: INFO: PersistentVolumeClaim pvc-cwjgv found and phase=Bound (4.307952s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:50:59.052[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:50:59.154[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:50:59.271[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:50:59.271[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:50:59.271[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:50:59.378[0m Jan 28 16:50:59.378: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-wn49t" in namespace "azuredisk-3612" to be "Succeeded or Failed" Jan 28 16:50:59.480: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 102.01235ms Jan 28 16:51:01.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205955768s Jan 28 16:51:03.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 4.206587899s Jan 28 16:51:05.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206216222s Jan 28 16:51:07.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206700851s Jan 28 16:51:09.582: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 10.204658354s Jan 28 16:51:11.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 12.206354177s Jan 28 16:51:13.584: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205772929s Jan 28 16:51:15.591: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Pending", Reason="", readiness=false. Elapsed: 16.212727937s Jan 28 16:51:17.590: INFO: Pod "azuredisk-volume-tester-wn49t": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.211854133s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:51:17.59[0m Jan 28 16:51:17.590: INFO: Pod "azuredisk-volume-tester-wn49t" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/28/23 16:51:17.59[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/28/23 16:51:22.591[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 16:51:22.795[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 16:51:22.795[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m01/28/23 16:51:22.905[0m Jan 28 16:51:22.905: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-5pfg4] to have phase Bound ... skipping 2 lines ... Jan 28 16:51:27.213: INFO: PersistentVolumeClaim pvc-5pfg4 found and phase=Bound (4.308476864s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:51:27.213[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:51:27.317[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:51:27.419[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:51:27.419[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/28/23 16:51:27.419[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:51:27.524[0m Jan 28 16:51:27.524: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-766w8" in namespace "azuredisk-3612" to be "Succeeded or Failed" Jan 28 16:51:27.625: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 101.616058ms Jan 28 16:51:29.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204044208s Jan 28 16:51:31.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204520474s Jan 28 16:51:33.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 6.204242889s Jan 28 16:51:35.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 8.204511453s Jan 28 16:51:37.729: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205510069s ... skipping 2 lines ... Jan 28 16:51:43.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 16.204442385s Jan 28 16:51:45.728: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 18.204711405s Jan 28 16:51:47.737: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 20.213376437s Jan 28 16:51:49.735: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Pending", Reason="", readiness=false. Elapsed: 22.211212966s Jan 28 16:51:51.735: INFO: Pod "azuredisk-volume-tester-766w8": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.211455012s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:51:51.735[0m Jan 28 16:51:51.735: INFO: Pod "azuredisk-volume-tester-766w8" satisfied condition "Succeeded or Failed" Jan 28 16:51:51.735: INFO: deleting Pod "azuredisk-3612"/"azuredisk-volume-tester-766w8" Jan 28 16:51:51.856: INFO: Pod azuredisk-volume-tester-766w8 has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-766w8 in namespace azuredisk-3612 [38;5;243m01/28/23 16:51:51.856[0m Jan 28 16:51:51.984: INFO: deleting PVC "azuredisk-3612"/"pvc-5pfg4" Jan 28 16:51:51.984: INFO: Deleting PersistentVolumeClaim "pvc-5pfg4" ... skipping 50 lines ... Jan 28 16:52:50.051: INFO: PersistentVolumeClaim pvc-rlnjf found and phase=Bound (4.30885864s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:52:50.051[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:52:50.153[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:52:50.255[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:52:50.255[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:52:50.255[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:52:50.362[0m Jan 28 16:52:50.362: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-k7q97" in namespace "azuredisk-8076" to be "Succeeded or Failed" Jan 28 16:52:50.464: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 101.836369ms Jan 28 16:52:52.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205746679s Jan 28 16:52:54.567: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204798611s Jan 28 16:52:56.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206326649s Jan 28 16:52:58.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206095629s Jan 28 16:53:00.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 10.206143919s Jan 28 16:53:02.567: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 12.20545822s Jan 28 16:53:04.569: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 14.206724668s Jan 28 16:53:06.575: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 16.212943357s Jan 28 16:53:08.574: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.212410339s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:53:08.575[0m Jan 28 16:53:08.575: INFO: Pod "azuredisk-volume-tester-k7q97" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/28/23 16:53:08.575[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/28/23 16:53:13.578[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 16:53:13.783[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 16:53:13.783[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m01/28/23 16:53:13.886[0m Jan 28 16:53:13.886: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-sgw4k] to have phase Bound ... skipping 2 lines ... Jan 28 16:53:18.196: INFO: PersistentVolumeClaim pvc-sgw4k found and phase=Bound (4.309978607s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:53:18.196[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:53:18.298[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:53:18.4[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:53:18.4[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/28/23 16:53:18.4[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:53:18.504[0m Jan 28 16:53:18.504: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-frr64" in namespace "azuredisk-8076" to be "Succeeded or Failed" Jan 28 16:53:18.606: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 101.525359ms Jan 28 16:53:20.709: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204699293s Jan 28 16:53:22.708: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 4.203889805s Jan 28 16:53:24.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206032012s Jan 28 16:53:26.709: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 8.204235231s Jan 28 16:53:28.708: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 10.203910062s Jan 28 16:53:30.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 12.205200127s Jan 28 16:53:32.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205722813s Jan 28 16:53:34.708: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 16.203965568s Jan 28 16:53:36.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 18.205360012s Jan 28 16:53:38.715: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 20.211005853s Jan 28 16:53:40.715: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Running", Reason="", readiness=false. Elapsed: 22.210613573s Jan 28 16:53:42.718: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Failed", Reason="", readiness=false. Elapsed: 24.21319679s Jan 28 16:53:42.718: INFO: Unexpected error: <*fmt.wrapError | 0xc000962060>: { msg: "error while waiting for pod azuredisk-8076/azuredisk-volume-tester-frr64 to be Succeeded or Failed: pod \"azuredisk-volume-tester-frr64\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.1.0.5 PodIP:192.168.103.147 PodIPs:[{IP:192.168.103.147}] StartTime:2023-01-28 16:53:18 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2023-01-28 16:53:38 +0000 UTC,FinishedAt:2023-01-28 16:53:38 +0000 UTC,ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053 Started:0xc000a3e81d}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", err: <*errors.errorString | 0xc000e64220>{ s: "pod \"azuredisk-volume-tester-frr64\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.1.0.5 PodIP:192.168.103.147 PodIPs:[{IP:192.168.103.147}] StartTime:2023-01-28 16:53:18 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2023-01-28 16:53:38 +0000 UTC,FinishedAt:2023-01-28 16:53:38 +0000 UTC,ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053 Started:0xc000a3e81d}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", }, } Jan 28 16:53:42.718: FAIL: error while waiting for pod azuredisk-8076/azuredisk-volume-tester-frr64 to be Succeeded or Failed: pod "azuredisk-volume-tester-frr64" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.1.0.5 PodIP:192.168.103.147 PodIPs:[{IP:192.168.103.147}] StartTime:2023-01-28 16:53:18 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2023-01-28 16:53:38 +0000 UTC,FinishedAt:2023-01-28 16:53:38 +0000 UTC,ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053 Started:0xc000a3e81d}] QOSClass:BestEffort EphemeralContainerStatuses:[]} Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x2253857?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeCloningTest).Run(0xc000895c28, {0x270dda0, 0xc000e2a680}, 0x13?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_cloning_tester.go:89 +0x5f4 ... skipping 30 lines ... Jan 28 16:54:34.723: INFO: Waiting up to 5m0s for PersistentVolumeClaim azuredisk-8076 to be removed Jan 28 16:54:34.825: INFO: Claim "azuredisk-8076" in namespace "pvc-rlnjf" doesn't exist in the system Jan 28 16:54:34.825: INFO: deleting StorageClass azuredisk-8076-disk.csi.azure.com-dynamic-sc-jbssx [1mSTEP:[0m dump namespace information after failure [38;5;243m01/28/23 16:54:34.929[0m [1mSTEP:[0m Destroying namespace "azuredisk-8076" for this suite. [38;5;243m01/28/23 16:54:34.93[0m [38;5;243m------------------------------[0m [38;5;9m• [FAILED] [110.946 seconds][0m Dynamic Provisioning [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:41[0m [single-az] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44[0m [38;5;9m[1m[It] should clone a volume of larger size than the source volume and make sure the filesystem is appropriately adjusted [disk.csi.azure.com][0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:572[0m ... skipping 16 lines ... Jan 28 16:52:50.051: INFO: PersistentVolumeClaim pvc-rlnjf found and phase=Bound (4.30885864s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:52:50.051[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:52:50.153[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:52:50.255[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:52:50.255[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:52:50.255[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:52:50.362[0m Jan 28 16:52:50.362: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-k7q97" in namespace "azuredisk-8076" to be "Succeeded or Failed" Jan 28 16:52:50.464: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 101.836369ms Jan 28 16:52:52.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205746679s Jan 28 16:52:54.567: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204798611s Jan 28 16:52:56.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206326649s Jan 28 16:52:58.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 8.206095629s Jan 28 16:53:00.568: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 10.206143919s Jan 28 16:53:02.567: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 12.20545822s Jan 28 16:53:04.569: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 14.206724668s Jan 28 16:53:06.575: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Pending", Reason="", readiness=false. Elapsed: 16.212943357s Jan 28 16:53:08.574: INFO: Pod "azuredisk-volume-tester-k7q97": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.212410339s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:53:08.575[0m Jan 28 16:53:08.575: INFO: Pod "azuredisk-volume-tester-k7q97" satisfied condition "Succeeded or Failed" [1mSTEP:[0m sleep 5s and then clone volume [38;5;243m01/28/23 16:53:08.575[0m [1mSTEP:[0m cloning existing volume [38;5;243m01/28/23 16:53:13.578[0m [1mSTEP:[0m setting up the PVC and PV [38;5;243m01/28/23 16:53:13.783[0m [1mSTEP:[0m creating a PVC [38;5;243m01/28/23 16:53:13.783[0m [1mSTEP:[0m waiting for PVC to be in phase "Bound" [38;5;243m01/28/23 16:53:13.886[0m Jan 28 16:53:13.886: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-sgw4k] to have phase Bound ... skipping 2 lines ... Jan 28 16:53:18.196: INFO: PersistentVolumeClaim pvc-sgw4k found and phase=Bound (4.309978607s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:53:18.196[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:53:18.298[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:53:18.4[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:53:18.4[0m [1mSTEP:[0m deploying a second pod with cloned volume [38;5;243m01/28/23 16:53:18.4[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:53:18.504[0m Jan 28 16:53:18.504: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-frr64" in namespace "azuredisk-8076" to be "Succeeded or Failed" Jan 28 16:53:18.606: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 101.525359ms Jan 28 16:53:20.709: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204699293s Jan 28 16:53:22.708: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 4.203889805s Jan 28 16:53:24.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206032012s Jan 28 16:53:26.709: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 8.204235231s Jan 28 16:53:28.708: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 10.203910062s Jan 28 16:53:30.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 12.205200127s Jan 28 16:53:32.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205722813s Jan 28 16:53:34.708: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 16.203965568s Jan 28 16:53:36.710: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 18.205360012s Jan 28 16:53:38.715: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Pending", Reason="", readiness=false. Elapsed: 20.211005853s Jan 28 16:53:40.715: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Running", Reason="", readiness=false. Elapsed: 22.210613573s Jan 28 16:53:42.718: INFO: Pod "azuredisk-volume-tester-frr64": Phase="Failed", Reason="", readiness=false. Elapsed: 24.21319679s Jan 28 16:53:42.718: INFO: Unexpected error: <*fmt.wrapError | 0xc000962060>: { msg: "error while waiting for pod azuredisk-8076/azuredisk-volume-tester-frr64 to be Succeeded or Failed: pod \"azuredisk-volume-tester-frr64\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.1.0.5 PodIP:192.168.103.147 PodIPs:[{IP:192.168.103.147}] StartTime:2023-01-28 16:53:18 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2023-01-28 16:53:38 +0000 UTC,FinishedAt:2023-01-28 16:53:38 +0000 UTC,ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053 Started:0xc000a3e81d}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", err: <*errors.errorString | 0xc000e64220>{ s: "pod \"azuredisk-volume-tester-frr64\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.1.0.5 PodIP:192.168.103.147 PodIPs:[{IP:192.168.103.147}] StartTime:2023-01-28 16:53:18 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2023-01-28 16:53:38 +0000 UTC,FinishedAt:2023-01-28 16:53:38 +0000 UTC,ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053 Started:0xc000a3e81d}] QOSClass:BestEffort EphemeralContainerStatuses:[]}", }, } Jan 28 16:53:42.718: FAIL: error while waiting for pod azuredisk-8076/azuredisk-volume-tester-frr64 to be Succeeded or Failed: pod "azuredisk-volume-tester-frr64" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.1.0.5 PodIP:192.168.103.147 PodIPs:[{IP:192.168.103.147}] StartTime:2023-01-28 16:53:18 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2023-01-28 16:53:38 +0000 UTC,FinishedAt:2023-01-28 16:53:38 +0000 UTC,ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053 Started:0xc000a3e81d}] QOSClass:BestEffort EphemeralContainerStatuses:[]} Full Stack Trace sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*TestPod).WaitForSuccess(0x2253857?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823 +0x5d sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites.(*DynamicallyProvisionedVolumeCloningTest).Run(0xc000895c28, {0x270dda0, 0xc000e2a680}, 0x13?) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/dynamically_provisioned_volume_cloning_tester.go:89 +0x5f4 ... skipping 31 lines ... Jan 28 16:54:34.825: INFO: Claim "azuredisk-8076" in namespace "pvc-rlnjf" doesn't exist in the system Jan 28 16:54:34.825: INFO: deleting StorageClass azuredisk-8076-disk.csi.azure.com-dynamic-sc-jbssx [1mSTEP:[0m dump namespace information after failure [38;5;243m01/28/23 16:54:34.929[0m [1mSTEP:[0m Destroying namespace "azuredisk-8076" for this suite. [38;5;243m01/28/23 16:54:34.93[0m [38;5;243m<< End Captured GinkgoWriter Output[0m [38;5;9mJan 28 16:53:42.718: error while waiting for pod azuredisk-8076/azuredisk-volume-tester-frr64 to be Succeeded or Failed: pod "azuredisk-volume-tester-frr64" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:39 +0000 UTC Reason:PodFailed Message:} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2023-01-28 16:53:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.1.0.5 PodIP:192.168.103.147 PodIPs:[{IP:192.168.103.147}] StartTime:2023-01-28 16:53:18 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2023-01-28 16:53:38 +0000 UTC,FinishedAt:2023-01-28 16:53:38 +0000 UTC,ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:registry.k8s.io/e2e-test-images/busybox:1.29-4 ImageID:registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 ContainerID:containerd://d1fe3058e87271d6b5af13855b44d0f90a1c54f2e7312536f1fac17d30f13053 Started:0xc000a3e81d}] QOSClass:BestEffort EphemeralContainerStatuses:[]}[0m [38;5;9mIn [1m[It][0m[38;5;9m at: [1m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [1mThere were additional failures detected after the initial failure:[0m [38;5;13m[PANICKED][0m [38;5;13mTest Panicked[0m [38;5;13mIn [1m[DeferCleanup (Each)][0m[38;5;13m at: [1m/usr/local/go/src/runtime/panic.go:260[0m [38;5;13mruntime error: invalid memory address or nil pointer dereference[0m [38;5;13mFull Stack Trace[0m k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1() /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:274 +0x5c k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0000322d0) /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:271 +0x179 ... skipping 52 lines ... Jan 28 16:54:50.460: INFO: PersistentVolumeClaim pvc-8rjp9 found and phase=Bound (4.30882665s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:54:50.46[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:54:50.562[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:54:50.664[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:54:50.664[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:54:50.664[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:54:50.769[0m Jan 28 16:54:50.769: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-hsbhq" in namespace "azuredisk-4801" to be "Succeeded or Failed" Jan 28 16:54:50.879: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 109.898665ms Jan 28 16:54:52.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220763552s Jan 28 16:54:54.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 4.220765483s Jan 28 16:54:56.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 6.2211928s Jan 28 16:54:58.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 8.220981389s Jan 28 16:55:00.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 10.220228876s ... skipping 6 lines ... Jan 28 16:55:14.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 24.220267105s Jan 28 16:55:16.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 26.220130178s Jan 28 16:55:18.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Running", Reason="", readiness=true. Elapsed: 28.220037217s Jan 28 16:55:20.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Running", Reason="", readiness=false. Elapsed: 30.220279511s Jan 28 16:55:22.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Succeeded", Reason="", readiness=false. Elapsed: 32.220088165s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:55:22.989[0m Jan 28 16:55:22.989: INFO: Pod "azuredisk-volume-tester-hsbhq" satisfied condition "Succeeded or Failed" Jan 28 16:55:22.989: INFO: deleting Pod "azuredisk-4801"/"azuredisk-volume-tester-hsbhq" Jan 28 16:55:23.094: INFO: Pod azuredisk-volume-tester-hsbhq has the following logs: hello world hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-hsbhq in namespace azuredisk-4801 [38;5;243m01/28/23 16:55:23.095[0m ... skipping 87 lines ... Jan 28 16:54:50.460: INFO: PersistentVolumeClaim pvc-8rjp9 found and phase=Bound (4.30882665s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:54:50.46[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:54:50.562[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:54:50.664[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:54:50.664[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:54:50.664[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:54:50.769[0m Jan 28 16:54:50.769: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-hsbhq" in namespace "azuredisk-4801" to be "Succeeded or Failed" Jan 28 16:54:50.879: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 109.898665ms Jan 28 16:54:52.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220763552s Jan 28 16:54:54.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 4.220765483s Jan 28 16:54:56.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 6.2211928s Jan 28 16:54:58.990: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 8.220981389s Jan 28 16:55:00.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 10.220228876s ... skipping 6 lines ... Jan 28 16:55:14.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 24.220267105s Jan 28 16:55:16.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Pending", Reason="", readiness=false. Elapsed: 26.220130178s Jan 28 16:55:18.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Running", Reason="", readiness=true. Elapsed: 28.220037217s Jan 28 16:55:20.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Running", Reason="", readiness=false. Elapsed: 30.220279511s Jan 28 16:55:22.989: INFO: Pod "azuredisk-volume-tester-hsbhq": Phase="Succeeded", Reason="", readiness=false. Elapsed: 32.220088165s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:55:22.989[0m Jan 28 16:55:22.989: INFO: Pod "azuredisk-volume-tester-hsbhq" satisfied condition "Succeeded or Failed" Jan 28 16:55:22.989: INFO: deleting Pod "azuredisk-4801"/"azuredisk-volume-tester-hsbhq" Jan 28 16:55:23.094: INFO: Pod azuredisk-volume-tester-hsbhq has the following logs: hello world hello world hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-hsbhq in namespace azuredisk-4801 [38;5;243m01/28/23 16:55:23.095[0m ... skipping 73 lines ... Jan 28 16:56:36.487: INFO: PersistentVolumeClaim pvc-drzqh found and phase=Bound (4.310272167s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:56:36.487[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:56:36.589[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:56:36.692[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:56:36.692[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:56:36.692[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:56:36.797[0m Jan 28 16:56:36.797: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-pqmbg" in namespace "azuredisk-8154" to be "Succeeded or Failed" Jan 28 16:56:36.904: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 106.50165ms Jan 28 16:56:39.008: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 2.210955929s Jan 28 16:56:41.006: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 4.209400968s Jan 28 16:56:43.009: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 6.211566246s Jan 28 16:56:45.008: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 8.210871269s Jan 28 16:56:47.007: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 10.210235881s ... skipping 4 lines ... Jan 28 16:56:57.007: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 20.21018052s Jan 28 16:56:59.008: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 22.21071186s Jan 28 16:57:01.015: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 24.217494936s Jan 28 16:57:03.014: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 26.216855801s Jan 28 16:57:05.014: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Succeeded", Reason="", readiness=false. Elapsed: 28.216859122s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:57:05.014[0m Jan 28 16:57:05.014: INFO: Pod "azuredisk-volume-tester-pqmbg" satisfied condition "Succeeded or Failed" Jan 28 16:57:05.014: INFO: deleting Pod "azuredisk-8154"/"azuredisk-volume-tester-pqmbg" Jan 28 16:57:05.138: INFO: Pod azuredisk-volume-tester-pqmbg has the following logs: 100+0 records in 100+0 records out 104857600 bytes (100.0MB) copied, 0.060401 seconds, 1.6GB/s hello world ... skipping 65 lines ... Jan 28 16:56:36.487: INFO: PersistentVolumeClaim pvc-drzqh found and phase=Bound (4.310272167s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:56:36.487[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:56:36.589[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:56:36.692[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:56:36.692[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:56:36.692[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:56:36.797[0m Jan 28 16:56:36.797: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-pqmbg" in namespace "azuredisk-8154" to be "Succeeded or Failed" Jan 28 16:56:36.904: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 106.50165ms Jan 28 16:56:39.008: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 2.210955929s Jan 28 16:56:41.006: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 4.209400968s Jan 28 16:56:43.009: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 6.211566246s Jan 28 16:56:45.008: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 8.210871269s Jan 28 16:56:47.007: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 10.210235881s ... skipping 4 lines ... Jan 28 16:56:57.007: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 20.21018052s Jan 28 16:56:59.008: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 22.21071186s Jan 28 16:57:01.015: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 24.217494936s Jan 28 16:57:03.014: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Pending", Reason="", readiness=false. Elapsed: 26.216855801s Jan 28 16:57:05.014: INFO: Pod "azuredisk-volume-tester-pqmbg": Phase="Succeeded", Reason="", readiness=false. Elapsed: 28.216859122s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:57:05.014[0m Jan 28 16:57:05.014: INFO: Pod "azuredisk-volume-tester-pqmbg" satisfied condition "Succeeded or Failed" Jan 28 16:57:05.014: INFO: deleting Pod "azuredisk-8154"/"azuredisk-volume-tester-pqmbg" Jan 28 16:57:05.138: INFO: Pod azuredisk-volume-tester-pqmbg has the following logs: 100+0 records in 100+0 records out 104857600 bytes (100.0MB) copied, 0.060401 seconds, 1.6GB/s hello world ... skipping 50 lines ... Jan 28 16:58:01.080: INFO: PersistentVolumeClaim pvc-9v89z found but phase is Pending instead of Bound. Jan 28 16:58:03.183: INFO: PersistentVolumeClaim pvc-9v89z found and phase=Bound (4.309230371s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:58:03.183[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:58:03.287[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:58:03.389[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:58:03.39[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:58:03.493[0m Jan 28 16:58:03.493: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-k4jrz" in namespace "azuredisk-1166" to be "Succeeded or Failed" Jan 28 16:58:03.595: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 101.879553ms Jan 28 16:58:05.697: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204571564s Jan 28 16:58:07.697: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204399358s Jan 28 16:58:09.699: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 6.205736438s Jan 28 16:58:11.703: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 8.210249259s Jan 28 16:58:13.704: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Running", Reason="", readiness=true. Elapsed: 10.211318792s Jan 28 16:58:15.705: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Running", Reason="", readiness=false. Elapsed: 12.21244061s Jan 28 16:58:17.709: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 14.216267984s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:58:17.709[0m Jan 28 16:58:17.709: INFO: Pod "azuredisk-volume-tester-k4jrz" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 16:58:17.709[0m [1mSTEP:[0m Prow test resource group: capz-v4slna [38;5;243m01/28/23 16:58:17.71[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-f63fd724-9f2c-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 16:58:17.71[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-f63fd724-9f2c-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 16:58:19.906[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/28/23 16:58:19.906[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/28/23 16:58:19.906[0m ... skipping 11 lines ... Jan 28 16:58:39.744: INFO: PersistentVolumeClaim pvc-pr9j2 found and phase=Bound (4.308569361s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:58:39.744[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:58:39.846[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:58:39.949[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:58:39.949[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 16:58:39.949[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:58:40.053[0m Jan 28 16:58:40.053: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-p594l" in namespace "azuredisk-1166" to be "Succeeded or Failed" Jan 28 16:58:40.155: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 102.072282ms Jan 28 16:58:42.258: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205358668s Jan 28 16:58:44.258: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204676786s Jan 28 16:58:46.260: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206558845s Jan 28 16:58:48.262: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209344644s Jan 28 16:58:50.259: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205991342s Jan 28 16:58:52.260: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 12.206660262s Jan 28 16:58:54.265: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Running", Reason="", readiness=true. Elapsed: 14.211705863s Jan 28 16:58:56.267: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Running", Reason="", readiness=false. Elapsed: 16.213956713s Jan 28 16:58:58.267: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.213491415s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:58:58.267[0m Jan 28 16:58:58.267: INFO: Pod "azuredisk-volume-tester-p594l" satisfied condition "Succeeded or Failed" Jan 28 16:58:58.267: INFO: deleting Pod "azuredisk-1166"/"azuredisk-volume-tester-p594l" Jan 28 16:58:58.382: INFO: Pod azuredisk-volume-tester-p594l has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-p594l in namespace azuredisk-1166 [38;5;243m01/28/23 16:58:58.382[0m Jan 28 16:58:58.499: INFO: deleting PVC "azuredisk-1166"/"pvc-pr9j2" Jan 28 16:58:58.499: INFO: Deleting PersistentVolumeClaim "pvc-pr9j2" ... skipping 50 lines ... Jan 28 16:58:01.080: INFO: PersistentVolumeClaim pvc-9v89z found but phase is Pending instead of Bound. Jan 28 16:58:03.183: INFO: PersistentVolumeClaim pvc-9v89z found and phase=Bound (4.309230371s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:58:03.183[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:58:03.287[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:58:03.389[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 16:58:03.39[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:58:03.493[0m Jan 28 16:58:03.493: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-k4jrz" in namespace "azuredisk-1166" to be "Succeeded or Failed" Jan 28 16:58:03.595: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 101.879553ms Jan 28 16:58:05.697: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204571564s Jan 28 16:58:07.697: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204399358s Jan 28 16:58:09.699: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 6.205736438s Jan 28 16:58:11.703: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Pending", Reason="", readiness=false. Elapsed: 8.210249259s Jan 28 16:58:13.704: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Running", Reason="", readiness=true. Elapsed: 10.211318792s Jan 28 16:58:15.705: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Running", Reason="", readiness=false. Elapsed: 12.21244061s Jan 28 16:58:17.709: INFO: Pod "azuredisk-volume-tester-k4jrz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 14.216267984s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:58:17.709[0m Jan 28 16:58:17.709: INFO: Pod "azuredisk-volume-tester-k4jrz" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 16:58:17.709[0m [1mSTEP:[0m Prow test resource group: capz-v4slna [38;5;243m01/28/23 16:58:17.71[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-f63fd724-9f2c-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 16:58:17.71[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-f63fd724-9f2c-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 16:58:19.906[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/28/23 16:58:19.906[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/28/23 16:58:19.906[0m ... skipping 11 lines ... Jan 28 16:58:39.744: INFO: PersistentVolumeClaim pvc-pr9j2 found and phase=Bound (4.308569361s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 16:58:39.744[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 16:58:39.846[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 16:58:39.949[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 16:58:39.949[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 16:58:39.949[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 16:58:40.053[0m Jan 28 16:58:40.053: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-p594l" in namespace "azuredisk-1166" to be "Succeeded or Failed" Jan 28 16:58:40.155: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 102.072282ms Jan 28 16:58:42.258: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205358668s Jan 28 16:58:44.258: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 4.204676786s Jan 28 16:58:46.260: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206558845s Jan 28 16:58:48.262: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 8.209344644s Jan 28 16:58:50.259: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205991342s Jan 28 16:58:52.260: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Pending", Reason="", readiness=false. Elapsed: 12.206660262s Jan 28 16:58:54.265: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Running", Reason="", readiness=true. Elapsed: 14.211705863s Jan 28 16:58:56.267: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Running", Reason="", readiness=false. Elapsed: 16.213956713s Jan 28 16:58:58.267: INFO: Pod "azuredisk-volume-tester-p594l": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.213491415s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 16:58:58.267[0m Jan 28 16:58:58.267: INFO: Pod "azuredisk-volume-tester-p594l" satisfied condition "Succeeded or Failed" Jan 28 16:58:58.267: INFO: deleting Pod "azuredisk-1166"/"azuredisk-volume-tester-p594l" Jan 28 16:58:58.382: INFO: Pod azuredisk-volume-tester-p594l has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-p594l in namespace azuredisk-1166 [38;5;243m01/28/23 16:58:58.382[0m Jan 28 16:58:58.499: INFO: deleting PVC "azuredisk-1166"/"pvc-pr9j2" Jan 28 16:58:58.499: INFO: Deleting PersistentVolumeClaim "pvc-pr9j2" ... skipping 49 lines ... Jan 28 17:00:52.310: INFO: PersistentVolumeClaim pvc-466jp found but phase is Pending instead of Bound. Jan 28 17:00:54.413: INFO: PersistentVolumeClaim pvc-466jp found and phase=Bound (4.309762339s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:00:54.413[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:00:54.515[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:00:54.619[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 17:00:54.619[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:00:54.722[0m Jan 28 17:00:54.723: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-ljwd2" in namespace "azuredisk-4415" to be "Succeeded or Failed" Jan 28 17:00:54.825: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 102.211391ms Jan 28 17:00:56.929: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205826879s Jan 28 17:00:58.929: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.205858946s Jan 28 17:01:00.928: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.204925s Jan 28 17:01:02.927: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.2047655s Jan 28 17:01:04.927: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.20473844s Jan 28 17:01:06.928: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 12.205260914s Jan 28 17:01:08.928: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205769885s Jan 28 17:01:10.935: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 16.211924002s Jan 28 17:01:12.937: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.214055103s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:01:12.937[0m Jan 28 17:01:12.937: INFO: Pod "azuredisk-volume-tester-ljwd2" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 17:01:12.937[0m [1mSTEP:[0m Prow test resource group: capz-v4slna [38;5;243m01/28/23 17:01:12.938[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-5eb174e0-9f2d-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 17:01:12.938[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-5eb174e0-9f2d-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 17:01:13.73[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/28/23 17:01:13.73[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/28/23 17:01:13.73[0m ... skipping 22 lines ... [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:01:43.904[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:01:44.007[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m Set pod anti-affinity to make sure two pods are scheduled on different nodes [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:01:44.214[0m Jan 28 17:01:44.214: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-hz58r" in namespace "azuredisk-4415" to be "Succeeded or Failed" Jan 28 17:01:44.316: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 101.732329ms Jan 28 17:01:46.419: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205220654s Jan 28 17:01:48.419: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 4.205159031s Jan 28 17:01:50.419: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 6.205041375s Jan 28 17:01:52.418: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 8.204538382s Jan 28 17:01:54.420: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205865344s Jan 28 17:01:56.420: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 12.206054552s Jan 28 17:01:58.427: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 14.212710102s Jan 28 17:02:00.427: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 16.213188196s Jan 28 17:02:02.427: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.212683054s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:02:02.427[0m Jan 28 17:02:02.427: INFO: Pod "azuredisk-volume-tester-hz58r" satisfied condition "Succeeded or Failed" Jan 28 17:02:02.427: INFO: deleting Pod "azuredisk-4415"/"azuredisk-volume-tester-hz58r" Jan 28 17:02:02.545: INFO: Pod azuredisk-volume-tester-hz58r has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-hz58r in namespace azuredisk-4415 [38;5;243m01/28/23 17:02:02.545[0m Jan 28 17:02:02.665: INFO: deleting PVC "azuredisk-4415"/"pvc-n4jn4" Jan 28 17:02:02.665: INFO: Deleting PersistentVolumeClaim "pvc-n4jn4" ... skipping 57 lines ... Jan 28 17:00:52.310: INFO: PersistentVolumeClaim pvc-466jp found but phase is Pending instead of Bound. Jan 28 17:00:54.413: INFO: PersistentVolumeClaim pvc-466jp found and phase=Bound (4.309762339s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:00:54.413[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:00:54.515[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:00:54.619[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 17:00:54.619[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:00:54.722[0m Jan 28 17:00:54.723: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-ljwd2" in namespace "azuredisk-4415" to be "Succeeded or Failed" Jan 28 17:00:54.825: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 102.211391ms Jan 28 17:00:56.929: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205826879s Jan 28 17:00:58.929: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.205858946s Jan 28 17:01:00.928: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.204925s Jan 28 17:01:02.927: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.2047655s Jan 28 17:01:04.927: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.20473844s Jan 28 17:01:06.928: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 12.205260914s Jan 28 17:01:08.928: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205769885s Jan 28 17:01:10.935: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Pending", Reason="", readiness=false. Elapsed: 16.211924002s Jan 28 17:01:12.937: INFO: Pod "azuredisk-volume-tester-ljwd2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.214055103s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:01:12.937[0m Jan 28 17:01:12.937: INFO: Pod "azuredisk-volume-tester-ljwd2" satisfied condition "Succeeded or Failed" [1mSTEP:[0m Checking Prow test resource group [38;5;243m01/28/23 17:01:12.937[0m [1mSTEP:[0m Prow test resource group: capz-v4slna [38;5;243m01/28/23 17:01:12.938[0m [1mSTEP:[0m Creating external resource group: azuredisk-csi-driver-test-5eb174e0-9f2d-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 17:01:12.938[0m [1mSTEP:[0m creating volume snapshot class with external rg azuredisk-csi-driver-test-5eb174e0-9f2d-11ed-a327-524ed4a8bfa9 [38;5;243m01/28/23 17:01:13.73[0m [1mSTEP:[0m setting up the VolumeSnapshotClass [38;5;243m01/28/23 17:01:13.73[0m [1mSTEP:[0m creating a VolumeSnapshotClass [38;5;243m01/28/23 17:01:13.73[0m ... skipping 22 lines ... [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:01:43.904[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:01:44.007[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m Set pod anti-affinity to make sure two pods are scheduled on different nodes [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m deploying a pod with a volume restored from the snapshot [38;5;243m01/28/23 17:01:44.11[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:01:44.214[0m Jan 28 17:01:44.214: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-hz58r" in namespace "azuredisk-4415" to be "Succeeded or Failed" Jan 28 17:01:44.316: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 101.732329ms Jan 28 17:01:46.419: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205220654s Jan 28 17:01:48.419: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 4.205159031s Jan 28 17:01:50.419: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 6.205041375s Jan 28 17:01:52.418: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 8.204538382s Jan 28 17:01:54.420: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 10.205865344s Jan 28 17:01:56.420: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 12.206054552s Jan 28 17:01:58.427: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 14.212710102s Jan 28 17:02:00.427: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Pending", Reason="", readiness=false. Elapsed: 16.213188196s Jan 28 17:02:02.427: INFO: Pod "azuredisk-volume-tester-hz58r": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.212683054s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:02:02.427[0m Jan 28 17:02:02.427: INFO: Pod "azuredisk-volume-tester-hz58r" satisfied condition "Succeeded or Failed" Jan 28 17:02:02.427: INFO: deleting Pod "azuredisk-4415"/"azuredisk-volume-tester-hz58r" Jan 28 17:02:02.545: INFO: Pod azuredisk-volume-tester-hz58r has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-hz58r in namespace azuredisk-4415 [38;5;243m01/28/23 17:02:02.545[0m Jan 28 17:02:02.665: INFO: deleting PVC "azuredisk-4415"/"pvc-n4jn4" Jan 28 17:02:02.665: INFO: Deleting PersistentVolumeClaim "pvc-n4jn4" ... skipping 80 lines ... Jan 28 17:04:28.344: INFO: PersistentVolumeClaim pvc-xjn6j found but phase is Pending instead of Bound. Jan 28 17:04:30.448: INFO: PersistentVolumeClaim pvc-xjn6j found and phase=Bound (4.312460677s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:04:30.448[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:04:30.55[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:04:30.652[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 17:04:30.653[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:04:30.759[0m Jan 28 17:04:30.759: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-h4szn" in namespace "azuredisk-6720" to be "Succeeded or Failed" Jan 28 17:04:30.868: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 108.997073ms Jan 28 17:04:32.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220230692s Jan 28 17:04:34.977: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.218237563s Jan 28 17:04:36.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.220454643s Jan 28 17:04:38.978: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.219442563s Jan 28 17:04:40.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.220249273s Jan 28 17:04:42.988: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 12.228819855s Jan 28 17:04:44.977: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 14.218428838s Jan 28 17:04:46.978: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 16.219348788s Jan 28 17:04:48.985: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 18.226400266s Jan 28 17:04:50.978: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 20.219345246s Jan 28 17:04:52.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Succeeded", Reason="", readiness=false. Elapsed: 22.219705112s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:04:52.979[0m Jan 28 17:04:52.979: INFO: Pod "azuredisk-volume-tester-h4szn" satisfied condition "Succeeded or Failed" Jan 28 17:04:52.979: INFO: deleting Pod "azuredisk-6720"/"azuredisk-volume-tester-h4szn" Jan 28 17:04:53.084: INFO: Pod azuredisk-volume-tester-h4szn has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-h4szn in namespace azuredisk-6720 [38;5;243m01/28/23 17:04:53.084[0m Jan 28 17:04:53.201: INFO: deleting PVC "azuredisk-6720"/"pvc-xjn6j" Jan 28 17:04:53.201: INFO: Deleting PersistentVolumeClaim "pvc-xjn6j" ... skipping 88 lines ... Jan 28 17:04:28.344: INFO: PersistentVolumeClaim pvc-xjn6j found but phase is Pending instead of Bound. Jan 28 17:04:30.448: INFO: PersistentVolumeClaim pvc-xjn6j found and phase=Bound (4.312460677s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:04:30.448[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:04:30.55[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:04:30.652[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 17:04:30.653[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:04:30.759[0m Jan 28 17:04:30.759: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-h4szn" in namespace "azuredisk-6720" to be "Succeeded or Failed" Jan 28 17:04:30.868: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 108.997073ms Jan 28 17:04:32.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220230692s Jan 28 17:04:34.977: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.218237563s Jan 28 17:04:36.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.220454643s Jan 28 17:04:38.978: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.219442563s Jan 28 17:04:40.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.220249273s Jan 28 17:04:42.988: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 12.228819855s Jan 28 17:04:44.977: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 14.218428838s Jan 28 17:04:46.978: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 16.219348788s Jan 28 17:04:48.985: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 18.226400266s Jan 28 17:04:50.978: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Pending", Reason="", readiness=false. Elapsed: 20.219345246s Jan 28 17:04:52.979: INFO: Pod "azuredisk-volume-tester-h4szn": Phase="Succeeded", Reason="", readiness=false. Elapsed: 22.219705112s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:04:52.979[0m Jan 28 17:04:52.979: INFO: Pod "azuredisk-volume-tester-h4szn" satisfied condition "Succeeded or Failed" Jan 28 17:04:52.979: INFO: deleting Pod "azuredisk-6720"/"azuredisk-volume-tester-h4szn" Jan 28 17:04:53.084: INFO: Pod azuredisk-volume-tester-h4szn has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-h4szn in namespace azuredisk-6720 [38;5;243m01/28/23 17:04:53.084[0m Jan 28 17:04:53.201: INFO: deleting PVC "azuredisk-6720"/"pvc-xjn6j" Jan 28 17:04:53.201: INFO: Deleting PersistentVolumeClaim "pvc-xjn6j" ... skipping 1175 lines ... Jan 28 17:27:02.074: INFO: PersistentVolumeClaim pvc-8hlfh found and phase=Bound (4.308081687s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:27:02.074[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:27:02.176[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:27:02.279[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 17:27:02.279[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 17:27:02.279[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:27:02.383[0m Jan 28 17:27:02.383: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fdfjm" in namespace "azuredisk-3173" to be "Succeeded or Failed" Jan 28 17:27:02.485: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 102.215976ms Jan 28 17:27:04.588: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205171976s Jan 28 17:27:06.591: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207930102s Jan 28 17:27:08.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20568279s Jan 28 17:27:10.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.205760767s Jan 28 17:27:12.588: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 10.204931242s Jan 28 17:27:14.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 12.205818332s Jan 28 17:27:16.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205950802s Jan 28 17:27:18.594: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 16.210549258s Jan 28 17:27:20.594: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.210987022s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:27:20.594[0m Jan 28 17:27:20.594: INFO: Pod "azuredisk-volume-tester-fdfjm" satisfied condition "Succeeded or Failed" Jan 28 17:27:20.594: INFO: deleting Pod "azuredisk-3173"/"azuredisk-volume-tester-fdfjm" Jan 28 17:27:20.714: INFO: Pod azuredisk-volume-tester-fdfjm has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-fdfjm in namespace azuredisk-3173 [38;5;243m01/28/23 17:27:20.714[0m Jan 28 17:27:20.833: INFO: deleting PVC "azuredisk-3173"/"pvc-8hlfh" Jan 28 17:27:20.833: INFO: Deleting PersistentVolumeClaim "pvc-8hlfh" ... skipping 39 lines ... Jan 28 17:27:02.074: INFO: PersistentVolumeClaim pvc-8hlfh found and phase=Bound (4.308081687s) [1mSTEP:[0m checking the PVC [38;5;243m01/28/23 17:27:02.074[0m [1mSTEP:[0m validating provisioned PV [38;5;243m01/28/23 17:27:02.176[0m [1mSTEP:[0m checking the PV [38;5;243m01/28/23 17:27:02.279[0m [1mSTEP:[0m setting up the pod [38;5;243m01/28/23 17:27:02.279[0m [1mSTEP:[0m deploying the pod [38;5;243m01/28/23 17:27:02.279[0m [1mSTEP:[0m checking that the pod's command exits with no error [38;5;243m01/28/23 17:27:02.383[0m Jan 28 17:27:02.383: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fdfjm" in namespace "azuredisk-3173" to be "Succeeded or Failed" Jan 28 17:27:02.485: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 102.215976ms Jan 28 17:27:04.588: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.205171976s Jan 28 17:27:06.591: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.207930102s Jan 28 17:27:08.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.20568279s Jan 28 17:27:10.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.205760767s Jan 28 17:27:12.588: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 10.204931242s Jan 28 17:27:14.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 12.205818332s Jan 28 17:27:16.589: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 14.205950802s Jan 28 17:27:18.594: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Pending", Reason="", readiness=false. Elapsed: 16.210549258s Jan 28 17:27:20.594: INFO: Pod "azuredisk-volume-tester-fdfjm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 18.210987022s [1mSTEP:[0m Saw pod success [38;5;243m01/28/23 17:27:20.594[0m Jan 28 17:27:20.594: INFO: Pod "azuredisk-volume-tester-fdfjm" satisfied condition "Succeeded or Failed" Jan 28 17:27:20.594: INFO: deleting Pod "azuredisk-3173"/"azuredisk-volume-tester-fdfjm" Jan 28 17:27:20.714: INFO: Pod azuredisk-volume-tester-fdfjm has the following logs: hello world [1mSTEP:[0m Deleting pod azuredisk-volume-tester-fdfjm in namespace azuredisk-3173 [38;5;243m01/28/23 17:27:20.714[0m Jan 28 17:27:20.833: INFO: deleting PVC "azuredisk-3173"/"pvc-8hlfh" Jan 28 17:27:20.833: INFO: Deleting PersistentVolumeClaim "pvc-8hlfh" ... skipping 85 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 16:23:13.604087 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 e2e-test I0128 16:23:13.604764 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 16:23:13.615033 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 16:23:13.615052 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 16:23:13.615060 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 16:23:13.615091 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 16:23:13.615649 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 16:23:13.615691 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 16:23:13.615750 1 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 I0128 16:23:13.615806 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0128 16:23:13.615874 1 azure.go:1007] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0128 16:23:13.615894 1 azuredisk.go:192] disable UseInstanceMetadata for controller I0128 16:23:13.615902 1 azuredisk.go:204] cloud: AzurePublicCloud, location: northeurope, rg: capz-v4slna, VMType: vmss, PrimaryScaleSetName: , PrimaryAvailabilitySetName: , DisableAvailabilitySetNodes: false I0128 16:23:13.622218 1 mount_linux.go:287] 'umount /tmp/kubelet-detect-safe-umount1022014224' failed with: exit status 32, output: umount: /tmp/kubelet-detect-safe-umount1022014224: must be superuser to unmount. I0128 16:23:13.622245 1 mount_linux.go:289] Detected umount with unsafe 'not mounted' behavior I0128 16:23:13.623239 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0128 16:23:13.623339 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0128 16:23:13.623408 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0128 16:23:13.623483 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0128 16:23:13.623580 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 144 lines ... I0128 16:26:49.736898 1 azure_controller_standard.go:236] azureDisk - update(capz-v4slna): vm(capz-v4slna-md-0-bq7hk) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/reattach-disk-multiple-nodes:reattach-disk-multiple-nodes]) returned with <nil> I0128 16:26:49.736954 1 azure_controller_standard.go:126] DeleteCacheForNode(capz-v4slna-md-0-bq7hk) successfully I0128 16:26:49.736971 1 azure_controller_standard.go:272] updateCache(capz-v4slna-md-0-bq7hk) successfully I0128 16:26:49.736988 1 azure_controller_common.go:422] azureDisk - detach disk(reattach-disk-multiple-nodes, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes) succeeded I0128 16:26:49.737021 1 controllerserver.go:425] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes to node capz-v4slna-md-0-s8npw again I0128 16:26:49.737078 1 azure_controller_common.go:194] found dangling volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes attached to node capz-v4slna-md-0-bq7hk, could not be attached to node(capz-v4slna-md-0-s8npw) E0128 16:26:49.737113 1 controllerserver.go:429] Attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes to instance capz-v4slna-md-0-s8npw failed with disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes) already attached to node(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/virtualMachines/capz-v4slna-md-0-bq7hk), could not be attached to node(capz-v4slna-md-0-s8npw) I0128 16:26:49.737226 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=15.349758751 request="azuredisk_csi_driver_controller_publish_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes" node="capz-v4slna-md-0-s8npw" result_code="failed_csi_driver_controller_publish_volume" E0128 16:26:49.737252 1 utils.go:82] GRPC error: rpc error: code = Internal desc = Attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes to instance capz-v4slna-md-0-s8npw failed with disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes) already attached to node(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/virtualMachines/capz-v4slna-md-0-bq7hk), could not be attached to node(capz-v4slna-md-0-s8npw) I0128 16:26:49.747945 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerPublishVolume I0128 16:26:49.747966 1 utils.go:78] GRPC request: {"node_id":"capz-v4slna-md-0-s8npw","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"cachingMode":"None","requestedsizegib":"10","skuName":"Premium_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes"} I0128 16:26:49.803394 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 973 I0128 16:26:49.803752 1 controllerserver.go:383] GetDiskLun returned: cannot find Lun for disk reattach-disk-multiple-nodes. Initiating attaching volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes to node capz-v4slna-md-0-s8npw (vmState Succeeded). I0128 16:26:49.803808 1 controllerserver.go:408] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes to node capz-v4slna-md-0-s8npw I0128 16:26:49.803852 1 azure_controller_common.go:255] Trying to attach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes lun 0 to node capz-v4slna-md-0-s8npw, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/reattach-disk-multiple-nodes:%!s(*provider.AttachDiskOptions=&{None reattach-disk-multiple-nodes false 0})] ... skipping 70 lines ... I0128 16:28:33.349922 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-02228b9b-4a20-47d2-81ba-b4597abf1bd0) returned with <nil> I0128 16:28:33.349983 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.392071265 request="azuredisk_csi_driver_controller_delete_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-02228b9b-4a20-47d2-81ba-b4597abf1bd0" result_code="succeeded" I0128 16:28:33.350010 1 utils.go:84] GRPC response: {} I0128 16:28:39.864393 1 utils.go:77] GRPC call: /csi.v1.Controller/CreateVolume I0128 16:28:39.864416 1 utils.go:78] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-a310735a-0fba-4349-9d5e-37f8ef3af4e3","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a310735a-0fba-4349-9d5e-37f8ef3af4e3","csi.storage.k8s.io/pvc/name":"pvc-cl8zr","csi.storage.k8s.io/pvc/namespace":"azuredisk-8266","enableAsyncAttach":"false","networkAccessPolicy":"DenyAll","skuName":"Standard_LRS","userAgent":"azuredisk-e2e-test"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]} I0128 16:28:39.865016 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 16:28:39.874382 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 16:28:39.874409 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 16:28:39.874417 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 16:28:39.874449 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 16:28:39.874692 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 16:28:39.874737 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 16:28:39.874779 1 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 ... skipping 41 lines ... I0128 16:29:39.289787 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-a310735a-0fba-4349-9d5e-37f8ef3af4e3) returned with <nil> I0128 16:29:39.289822 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.234400589 request="azuredisk_csi_driver_controller_delete_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-a310735a-0fba-4349-9d5e-37f8ef3af4e3" result_code="succeeded" I0128 16:29:39.289841 1 utils.go:84] GRPC response: {} I0128 16:29:45.949545 1 utils.go:77] GRPC call: /csi.v1.Controller/CreateVolume I0128 16:29:45.949579 1 utils.go:78] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":1099511627776},"name":"pvc-c4ca171c-e90d-4fc5-adf9-d6bb41ec9f03","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c4ca171c-e90d-4fc5-adf9-d6bb41ec9f03","csi.storage.k8s.io/pvc/name":"pvc-8xgtl","csi.storage.k8s.io/pvc/namespace":"azuredisk-4376","enableAsyncAttach":"false","enableBursting":"true","perfProfile":"Basic","skuName":"Premium_LRS","userAgent":"azuredisk-e2e-test"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":7}}]} I0128 16:29:45.950473 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 16:29:45.962265 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 16:29:45.962284 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 16:29:45.962290 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 16:29:45.962337 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 16:29:45.962692 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 16:29:45.962759 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 16:29:45.962849 1 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 ... skipping 333 lines ... I0128 16:42:16.149590 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume I0128 16:42:16.149607 1 utils.go:78] GRPC request: {"node_id":"capz-v4slna-md-0-s8npw","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8"} I0128 16:42:16.149717 1 controllerserver.go:471] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8 from node capz-v4slna-md-0-s8npw I0128 16:42:16.149762 1 azure_controller_common.go:398] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8 from node capz-v4slna-md-0-s8npw, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8:pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8] I0128 16:42:16.149786 1 azure_controller_standard.go:180] azureDisk - detach disk: name pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8 uri /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8 I0128 16:42:16.149796 1 azure_controller_standard.go:210] azureDisk - update(capz-v4slna): vm(capz-v4slna-md-0-s8npw) - detach disk list(capz-v4slna-md-0-s8npw)%!(EXTRA map[string]string=map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8:pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8]) I0128 16:42:16.151376 1 azure_armclient.go:301] Received error in sendAsync.send: resourceID: https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/virtualMachines/capz-v4slna-md-0-s8npw?api-version=2022-03-01, error: Retriable: true, RetryAfter: 0s, HTTPStatusCode: 503, RawError: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd"> <HTML><HEAD><TITLE>Service Unavailable</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Service Unavailable</h2> <hr><p>HTTP Error 503. The service is unavailable.</p> </BODY></HTML> I0128 16:42:16.151404 1 azure_armclient.go:482] Received error in patch.send: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/virtualMachines/capz-v4slna-md-0-s8npw, error: Retriable: true, RetryAfter: 0s, HTTPStatusCode: 503, RawError: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd"> <HTML><HEAD><TITLE>Service Unavailable</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Service Unavailable</h2> <hr><p>HTTP Error 503. The service is unavailable.</p> </BODY></HTML> I0128 16:42:16.151417 1 azure_vmclient.go:434] Received error in vm.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/virtualMachines/capz-v4slna-md-0-s8npw, error: Retriable: true, RetryAfter: 0s, HTTPStatusCode: 503, RawError: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd"> <HTML><HEAD><TITLE>Service Unavailable</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Service Unavailable</h2> <hr><p>HTTP Error 503. The service is unavailable.</p> </BODY></HTML> E0128 16:42:16.151458 1 azure_controller_standard.go:227] azureDisk - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8:pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8]) on rg(capz-v4slna) vm(capz-v4slna-md-0-s8npw) failed, err: &{true 503 0001-01-01 00:00:00 +0000 UTC <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd"> <HTML><HEAD><TITLE>Service Unavailable</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Service Unavailable</h2> <hr><p>HTTP Error 503. The service is unavailable.</p> </BODY></HTML> } I0128 16:42:16.151500 1 azure_controller_standard.go:236] azureDisk - update(capz-v4slna): vm(capz-v4slna-md-0-s8npw) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8:pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8]) returned with &{true 503 0001-01-01 00:00:00 +0000 UTC <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd"> <HTML><HEAD><TITLE>Service Unavailable</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Service Unavailable</h2> <hr><p>HTTP Error 503. The service is unavailable.</p> </BODY></HTML> } I0128 16:42:16.151522 1 azure_controller_standard.go:126] DeleteCacheForNode(capz-v4slna-md-0-s8npw) successfully E0128 16:42:16.151540 1 azure_controller_common.go:418] azureDisk - detach disk(pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8) failed, err: Retriable: true, RetryAfter: 0s, HTTPStatusCode: 503, RawError: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd"> <HTML><HEAD><TITLE>Service Unavailable</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Service Unavailable</h2> <hr><p>HTTP Error 503. The service is unavailable.</p> </BODY></HTML> I0128 16:42:16.151605 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=0.001850407 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8" node="capz-v4slna-md-0-s8npw" result_code="failed_csi_driver_controller_unpublish_volume" E0128 16:42:16.151622 1 utils.go:82] GRPC error: rpc error: code = Internal desc = Could not detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8 from node capz-v4slna-md-0-s8npw: Retriable: true, RetryAfter: 0s, HTTPStatusCode: 503, RawError: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd"> <HTML><HEAD><TITLE>Service Unavailable</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Service Unavailable</h2> <hr><p>HTTP Error 503. The service is unavailable.</p> </BODY></HTML> I0128 16:42:17.158636 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerUnpublishVolume I0128 16:42:17.158720 1 utils.go:78] GRPC request: {"node_id":"capz-v4slna-md-0-s8npw","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8"} I0128 16:42:17.158906 1 controllerserver.go:471] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8 from node capz-v4slna-md-0-s8npw I0128 16:42:17.210251 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 8446 I0128 16:42:17.211032 1 azure_controller_common.go:398] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8 from node capz-v4slna-md-0-s8npw, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8:pvc-5e410b9c-526b-42f3-8304-8e30bd7863e8] ... skipping 235 lines ... I0128 16:49:44.216101 1 azure_controller_common.go:398] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0 from node capz-v4slna-md-0-bq7hk, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0:pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0] E0128 16:49:44.216210 1 azure_controller_standard.go:189] detach azure disk on node(capz-v4slna-md-0-bq7hk): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0:pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0]) not found I0128 16:49:44.216233 1 azure_controller_standard.go:210] azureDisk - update(capz-v4slna): vm(capz-v4slna-md-0-bq7hk) - detach disk list(capz-v4slna-md-0-bq7hk)%!(EXTRA map[string]string=map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0:pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0]) I0128 16:49:47.775464 1 utils.go:77] GRPC call: /csi.v1.Controller/DeleteVolume I0128 16:49:47.775490 1 utils.go:78] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0"} I0128 16:49:47.775581 1 controllerserver.go:317] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0) I0128 16:49:47.775597 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0) since it's in attaching or detaching state I0128 16:49:47.775648 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=2.9301e-05 request="azuredisk_csi_driver_controller_delete_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0" result_code="failed_csi_driver_controller_delete_volume" E0128 16:49:47.775666 1 utils.go:82] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0) since it's in attaching or detaching state I0128 16:49:49.535778 1 azure_controller_standard.go:236] azureDisk - update(capz-v4slna): vm(capz-v4slna-md-0-bq7hk) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0:pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0]) returned with <nil> I0128 16:49:49.535837 1 azure_controller_standard.go:126] DeleteCacheForNode(capz-v4slna-md-0-bq7hk) successfully I0128 16:49:49.535854 1 azure_controller_standard.go:272] updateCache(capz-v4slna-md-0-bq7hk) successfully I0128 16:49:49.535871 1 azure_controller_common.go:422] azureDisk - detach disk(pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0) succeeded I0128 16:49:49.535891 1 controllerserver.go:480] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0 from node capz-v4slna-md-0-bq7hk successfully I0128 16:49:49.535953 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.319904408 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-0e2ffe2b-77e9-447b-aab0-0da555bf07e0" node="capz-v4slna-md-0-bq7hk" result_code="succeeded" ... skipping 829 lines ... I0128 17:05:19.787767 1 azure_controller_common.go:398] Trying to detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2 from node capz-v4slna-md-0-s8npw, diskMap: map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2:pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2] E0128 17:05:19.787796 1 azure_controller_standard.go:189] detach azure disk on node(capz-v4slna-md-0-s8npw): disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2:pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2]) not found I0128 17:05:19.787813 1 azure_controller_standard.go:210] azureDisk - update(capz-v4slna): vm(capz-v4slna-md-0-s8npw) - detach disk list(capz-v4slna-md-0-s8npw)%!(EXTRA map[string]string=map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2:pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2]) I0128 17:05:24.284736 1 utils.go:77] GRPC call: /csi.v1.Controller/DeleteVolume I0128 17:05:24.284779 1 utils.go:78] GRPC request: {"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2"} I0128 17:05:24.284999 1 controllerserver.go:317] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2) I0128 17:05:24.285059 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2) returned with failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2) since it's in attaching or detaching state I0128 17:05:24.285189 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=9.5906e-05 request="azuredisk_csi_driver_controller_delete_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2" result_code="failed_csi_driver_controller_delete_volume" E0128 17:05:24.285227 1 utils.go:82] GRPC error: failed to delete disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2) since it's in attaching or detaching state I0128 17:05:25.161226 1 azure_controller_standard.go:236] azureDisk - update(capz-v4slna): vm(capz-v4slna-md-0-s8npw) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/capz-v4slna/providers/microsoft.compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2:pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2]) returned with <nil> I0128 17:05:25.161357 1 azure_controller_standard.go:126] DeleteCacheForNode(capz-v4slna-md-0-s8npw) successfully I0128 17:05:25.161420 1 azure_controller_standard.go:272] updateCache(capz-v4slna-md-0-s8npw) successfully I0128 17:05:25.161462 1 azure_controller_common.go:422] azureDisk - detach disk(pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2) succeeded I0128 17:05:25.161508 1 controllerserver.go:480] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2 from node capz-v4slna-md-0-s8npw successfully I0128 17:05:25.161599 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.373910646 request="azuredisk_csi_driver_controller_unpublish_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-8cae2fbd-589c-4963-ae20-bc18c97cd5e2" node="capz-v4slna-md-0-s8npw" result_code="succeeded" ... skipping 152 lines ... I0128 17:10:46.755264 1 controllerserver.go:317] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ea22cbff-8995-4ab6-8d2d-988ab4e0df83) I0128 17:10:46.777990 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1217 I0128 17:10:51.991070 1 azure_managedDiskController.go:317] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ea22cbff-8995-4ab6-8d2d-988ab4e0df83 I0128 17:10:51.991101 1 controllerserver.go:319] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ea22cbff-8995-4ab6-8d2d-988ab4e0df83) returned with <nil> I0128 17:10:51.991146 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=5.235851148 request="azuredisk_csi_driver_controller_delete_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ea22cbff-8995-4ab6-8d2d-988ab4e0df83" result_code="succeeded" I0128 17:10:51.991163 1 utils.go:84] GRPC response: {} I0128 17:14:07.284285 1 azure_armclient.go:291] Received error in WaitForAsyncOperationCompletion: 'Future#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded' I0128 17:14:07.284418 1 azure_armclient.go:451] Received error in WaitForAsyncOperationResult: 'Future#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded', no response I0128 17:14:07.284446 1 azure_diskclient.go:273] Received error in disk.put.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-1827e571-199f-4fc6-a50b-117c58504d4c, error: Retriable: true, RetryAfter: 0s, HTTPStatusCode: -1, RawError: Future#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded I0128 17:14:07.284557 1 azure_metrics.go:115] "Observed Request Latency" latency_seconds=239.977462871 request="azuredisk_csi_driver_controller_expand_volume" resource_group="capz-v4slna" subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e" source="disk.csi.azure.com" volumeid="/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-1827e571-199f-4fc6-a50b-117c58504d4c" result_code="failed_csi_driver_controller_expand_volume" E0128 17:14:07.284577 1 utils.go:82] GRPC error: rpc error: code = Internal desc = failed to resize disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-1827e571-199f-4fc6-a50b-117c58504d4c) with error(Retriable: true, RetryAfter: 0s, HTTPStatusCode: -1, RawError: Future#WaitForCompletion: context has been cancelled: StatusCode=200 -- Original Error: context deadline exceeded) I0128 17:14:07.300258 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerGetCapabilities I0128 17:14:07.300278 1 utils.go:78] GRPC request: {} I0128 17:14:07.300330 1 utils.go:84] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":13}}}]} I0128 17:14:07.300880 1 utils.go:77] GRPC call: /csi.v1.Controller/ControllerExpandVolume I0128 17:14:07.301011 1 utils.go:78] GRPC request: {"capacity_range":{"required_bytes":21474836480},"volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-1827e571-199f-4fc6-a50b-117c58504d4c"} I0128 17:14:07.337693 1 util.go:124] Send.sendRequest got response with ContentLength -1, StatusCode 200 and responseBody length 1379 ... skipping 712 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 16:23:17.658285 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 e2e-test I0128 16:23:17.659529 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 16:23:17.682409 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 16:23:17.682439 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 16:23:17.682449 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 16:23:17.682498 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 16:23:17.683313 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 16:23:17.683369 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 16:23:17.683469 1 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 I0128 16:23:17.683538 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0128 16:23:17.683657 1 azure.go:1007] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0128 16:23:17.683710 1 azuredisk.go:192] disable UseInstanceMetadata for controller I0128 16:23:17.683721 1 azuredisk.go:204] cloud: AzurePublicCloud, location: northeurope, rg: capz-v4slna, VMType: vmss, PrimaryScaleSetName: , PrimaryAvailabilitySetName: , DisableAvailabilitySetNodes: false I0128 16:23:17.693577 1 mount_linux.go:287] 'umount /tmp/kubelet-detect-safe-umount2080430241' failed with: exit status 32, output: umount: /tmp/kubelet-detect-safe-umount2080430241: must be superuser to unmount. I0128 16:23:17.693620 1 mount_linux.go:289] Detected umount with unsafe 'not mounted' behavior I0128 16:23:17.693731 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0128 16:23:17.693747 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0128 16:23:17.693755 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0128 16:23:17.693762 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0128 16:23:17.693768 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 62 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 16:23:02.832062 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 e2e-test I0128 16:23:02.832507 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 16:23:02.843510 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 16:23:02.843554 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 16:23:02.843560 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 16:23:02.843584 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 16:23:02.844977 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 16:23:02.845035 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 16:23:02.845106 1 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 ... skipping 40 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 16:23:12.702778 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 e2e-test I0128 16:23:12.703372 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 16:23:12.805049 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 16:23:12.805076 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 16:23:12.805086 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 16:23:12.805139 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 16:23:12.805922 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 16:23:12.805997 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 16:23:12.806133 1 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 I0128 16:23:12.806237 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0128 16:23:12.806448 1 azure.go:1007] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0128 16:23:12.806544 1 skus.go:121] NewNodeInfo: Starting to populate node and disk sku information. I0128 16:23:12.828762 1 azure_instance_metadata.go:140] Warning: failed to get loadbalancer metadata: failure of getting loadbalancer metadata with response "404 Not Found" I0128 16:23:13.073607 1 mount_linux.go:284] Detected umount with safe 'not mounted' behavior I0128 16:23:13.073713 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0128 16:23:13.073728 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0128 16:23:13.073735 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0128 16:23:13.073741 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0128 16:23:13.073746 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 43 lines ... I0128 16:25:38.717294 1 utils.go:84] GRPC response: {} I0128 16:25:38.756911 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 16:25:38.756934 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/shared-disk-multiple-pods"} I0128 16:25:38.757073 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 I0128 16:25:38.757096 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 16:25:38.757105 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 I0128 16:25:38.759047 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 I0128 16:25:38.759067 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4" I0128 16:25:38.759184 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 successfully I0128 16:25:38.759239 1 utils.go:84] GRPC response: {} I0128 16:43:29.754758 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:43:29.754784 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/cb4c9f8589d54e4a444480336098e73546ba8adfb4bfccfaa7c36df7af64fa3e/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-955a5479-5f11-4a20-8c62-58ac3385f1a5","csi.storage.k8s.io/pvc/name":"pvc-ksqjj","csi.storage.k8s.io/pvc/namespace":"azuredisk-8582","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-955a5479-5f11-4a20-8c62-58ac3385f1a5"} I0128 16:43:31.555423 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 272 lines ... I0128 17:26:15.110792 1 utils.go:84] GRPC response: {} I0128 17:26:15.159722 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 17:26:15.159745 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf"} I0128 17:26:15.159827 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf I0128 17:26:15.159855 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 17:26:15.159869 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf I0128 17:26:15.161144 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf I0128 17:26:15.161163 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf" I0128 17:26:15.161283 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf successfully I0128 17:26:15.161303 1 utils.go:84] GRPC response: {} I0128 17:28:44.432818 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 17:28:44.432854 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/287d64cd9bf285f8ea6869987875410715969fb2d26f1367c1edad432c627d31/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-56e922e6-a372-4b8e-b00f-30a7363208c8","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-56e922e6-a372-4b8e-b00f-30a7363208c8"} I0128 17:28:46.215346 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 33 lines ... Platform: linux/amd64 Topology Key: topology.disk.csi.azure.com/zone Streaming logs below: I0128 16:23:09.101046 1 azuredisk.go:175] driver userAgent: disk.csi.azure.com/v1.27.0-8635ef7cb96ec669bd2a099af3b1437a19530391 e2e-test I0128 16:23:09.101595 1 azure_disk_utils.go:162] reading cloud config from secret kube-system/azure-cloud-provider I0128 16:23:09.115390 1 azure_disk_utils.go:169] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found I0128 16:23:09.115403 1 azure_disk_utils.go:174] could not read cloud config from secret kube-system/azure-cloud-provider I0128 16:23:09.115411 1 azure_disk_utils.go:184] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json I0128 16:23:09.115438 1 azure_disk_utils.go:192] read cloud config from file: /etc/kubernetes/azure.json successfully I0128 16:23:09.116058 1 azure_auth.go:253] Using AzurePublicCloud environment I0128 16:23:09.116088 1 azure_auth.go:138] azure: using client_id+client_secret to retrieve access token I0128 16:23:09.116169 1 azure_diskclient.go:68] Azure DisksClient using API version: 2022-03-02 I0128 16:23:09.116228 1 azure_blobclient.go:67] Azure BlobClient using API version: 2021-09-01 I0128 16:23:09.116283 1 azure.go:1007] attach/detach disk operation rate limit QPS: 6.000000, Bucket: 10 I0128 16:23:09.116300 1 skus.go:121] NewNodeInfo: Starting to populate node and disk sku information. I0128 16:23:09.145806 1 azure_instance_metadata.go:140] Warning: failed to get loadbalancer metadata: failure of getting loadbalancer metadata with response "404 Not Found" I0128 16:23:09.148217 1 mount_linux.go:284] Detected umount with safe 'not mounted' behavior I0128 16:23:09.148289 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME I0128 16:23:09.148301 1 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME I0128 16:23:09.148308 1 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT I0128 16:23:09.148314 1 driver.go:81] Enabling controller service capability: CLONE_VOLUME I0128 16:23:09.148320 1 driver.go:81] Enabling controller service capability: EXPAND_VOLUME ... skipping 84 lines ... I0128 16:25:38.567696 1 utils.go:84] GRPC response: {} I0128 16:25:38.619709 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 16:25:38.619744 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/shared-disk-multiple-pods"} I0128 16:25:38.619923 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 I0128 16:25:38.619971 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 16:25:38.620013 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 I0128 16:25:38.623365 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 I0128 16:25:38.623425 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4" I0128 16:25:38.623654 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/azuredisk-2790-disk.csi.azure.com-preprovisioned-pv-gm7r4 successfully I0128 16:25:38.623674 1 utils.go:84] GRPC response: {} I0128 16:27:00.466654 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:27:00.466681 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/50bbc955617390ed8d2cba6bff447d410f0419ab6688db4dd636be6fa6dd4cd6/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"cachingMode":"None","requestedsizegib":"10","skuName":"Premium_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/reattach-disk-multiple-nodes"} I0128 16:27:02.088392 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 168 lines ... I0128 16:31:41.803960 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:31:41.810965 1 mount_linux.go:570] Output: "" I0128 16:31:41.810990 1 mount_linux.go:529] Disk "/dev/disk/azure/scsi1/lun0" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/scsi1/lun0] I0128 16:31:42.303064 1 mount_linux.go:539] Disk successfully formatted (mkfs): ext4 - /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:31:42.303105 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:31:42.303150 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:31:42.334015 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:31:42.334068 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:31:42.858550 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:31:42.858568 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed2992de-bd14-4d3d-a766-85404c0f727f","csi.storage.k8s.io/pvc/name":"pvc-rg46b","csi.storage.k8s.io/pvc/namespace":"azuredisk-7996","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ed2992de-bd14-4d3d-a766-85404c0f727f"} I0128 16:31:44.478288 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:31:44.478320 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_LRS I0128 16:31:44.478781 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount with mount options([invalid mount options]) I0128 16:31:44.478813 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:31:44.484319 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 16:31:44.484339 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 16:31:44.494128 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:31:44.494172 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:31:44.518959 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:31:44.519226 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:31:45.583065 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:31:45.583086 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed2992de-bd14-4d3d-a766-85404c0f727f","csi.storage.k8s.io/pvc/name":"pvc-rg46b","csi.storage.k8s.io/pvc/namespace":"azuredisk-7996","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ed2992de-bd14-4d3d-a766-85404c0f727f"} I0128 16:31:47.191194 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:31:47.191234 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_LRS I0128 16:31:47.191617 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount with mount options([invalid mount options]) I0128 16:31:47.191643 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:31:47.197564 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 16:31:47.197583 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 16:31:47.207248 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:31:47.207289 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:31:47.222991 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:31:47.223030 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:31:49.317502 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:31:49.317525 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed2992de-bd14-4d3d-a766-85404c0f727f","csi.storage.k8s.io/pvc/name":"pvc-rg46b","csi.storage.k8s.io/pvc/namespace":"azuredisk-7996","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ed2992de-bd14-4d3d-a766-85404c0f727f"} I0128 16:31:50.911004 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:31:50.911044 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_LRS I0128 16:31:50.911430 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount with mount options([invalid mount options]) I0128 16:31:50.911456 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:31:50.917563 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 16:31:50.917585 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 16:31:50.927386 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:31:50.927417 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:31:50.955720 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:31:50.955767 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:31:54.988851 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:31:54.988879 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed2992de-bd14-4d3d-a766-85404c0f727f","csi.storage.k8s.io/pvc/name":"pvc-rg46b","csi.storage.k8s.io/pvc/namespace":"azuredisk-7996","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ed2992de-bd14-4d3d-a766-85404c0f727f"} I0128 16:31:56.646157 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:31:56.646200 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_LRS I0128 16:31:56.646591 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount with mount options([invalid mount options]) I0128 16:31:56.646642 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:31:56.665112 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 16:31:56.665132 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 16:31:56.681073 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:31:56.681103 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:31:56.697316 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:31:56.697353 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:32:04.717770 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:32:04.717799 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed2992de-bd14-4d3d-a766-85404c0f727f","csi.storage.k8s.io/pvc/name":"pvc-rg46b","csi.storage.k8s.io/pvc/namespace":"azuredisk-7996","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ed2992de-bd14-4d3d-a766-85404c0f727f"} I0128 16:32:06.316434 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:32:06.316483 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_LRS I0128 16:32:06.316866 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount with mount options([invalid mount options]) I0128 16:32:06.316899 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:32:06.321881 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 16:32:06.321904 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 16:32:06.330726 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:32:06.330778 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:32:06.360278 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:32:06.360560 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:32:22.473380 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:32:22.473399 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed2992de-bd14-4d3d-a766-85404c0f727f","csi.storage.k8s.io/pvc/name":"pvc-rg46b","csi.storage.k8s.io/pvc/namespace":"azuredisk-7996","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ed2992de-bd14-4d3d-a766-85404c0f727f"} I0128 16:32:24.061636 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:32:24.061753 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_LRS I0128 16:32:24.062458 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount with mount options([invalid mount options]) I0128 16:32:24.062484 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:32:24.067536 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 16:32:24.067557 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 16:32:24.077039 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:32:24.077075 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:32:24.095057 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:32:24.095311 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:32:56.163402 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:32:56.163426 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ed2992de-bd14-4d3d-a766-85404c0f727f","csi.storage.k8s.io/pvc/name":"pvc-rg46b","csi.storage.k8s.io/pvc/namespace":"azuredisk-7996","requestedsizegib":"10","skuName":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-ed2992de-bd14-4d3d-a766-85404c0f727f"} I0128 16:32:57.764631 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:32:57.764706 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_LRS I0128 16:32:57.765242 1 nodeserver.go:157] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount with mount options([invalid mount options]) I0128 16:32:57.765276 1 mount_linux.go:567] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0]) I0128 16:32:57.770189 1 mount_linux.go:570] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n" I0128 16:32:57.770208 1 mount_linux.go:453] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0 I0128 16:32:57.808333 1 mount_linux.go:557] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount I0128 16:32:57.808388 1 mount_linux.go:220] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount) E0128 16:32:57.840900 1 mount_linux.go:232] Mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. E0128 16:32:57.840944 1 utils.go:82] GRPC error: rpc error: code = Internal desc = could not format /dev/disk/azure/scsi1/lun0(lun: 0), and mount it at /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount, failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/2afee852324091dac77dc860faebaa1503e7148b1bc487e1ecf998646fda9d8b/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error. dmesg(1) may have more information after failed mount system call. I0128 16:34:26.452761 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:34:26.452782 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1","csi.storage.k8s.io/pvc/name":"pvc-vnkqr","csi.storage.k8s.io/pvc/namespace":"azuredisk-59","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1"} I0128 16:34:28.080535 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ I0128 16:34:28.080576 1 nodeserver.go:116] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType StandardSSD_ZRS I0128 16:34:28.080591 1 utils.go:84] GRPC response: {} I0128 16:34:28.091102 1 utils.go:77] GRPC call: /csi.v1.Node/NodePublishVolume ... skipping 16 lines ... I0128 16:34:33.325132 1 utils.go:84] GRPC response: {} I0128 16:34:33.375823 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 16:34:33.375858 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1"} I0128 16:34:33.375974 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1 I0128 16:34:33.376012 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 16:34:33.376043 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1 I0128 16:34:33.377918 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1 I0128 16:34:33.377944 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1" I0128 16:34:33.378325 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-db7688ae-67ff-49ed-9f39-7b282be75ae1 successfully I0128 16:34:33.378358 1 utils.go:84] GRPC response: {} I0128 16:35:30.534101 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:35:30.534305 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/f473da40484a15b3741f206d2f2caeeeba71fe015a0f10140d609aa67daf32c9/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-c8ff49f4-1fea-4cb9-a675-8ccbb4d4c03e","csi.storage.k8s.io/pvc/name":"pvc-vjwb5","csi.storage.k8s.io/pvc/namespace":"azuredisk-2546","requestedsizegib":"10","resourceGroup":"azuredisk-csi-driver-test-bfa1c3fa-9f29-11ed-a327-524ed4a8bfa9","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/azuredisk-csi-driver-test-bfa1c3fa-9f29-11ed-a327-524ed4a8bfa9/providers/Microsoft.Compute/disks/pvc-c8ff49f4-1fea-4cb9-a675-8ccbb4d4c03e"} I0128 16:35:32.102910 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 633 lines ... I0128 16:57:03.816277 1 utils.go:84] GRPC response: {} I0128 16:57:03.888999 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 16:57:03.889032 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a"} I0128 16:57:03.889143 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a I0128 16:57:03.889183 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 16:57:03.889210 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a I0128 16:57:03.891046 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a I0128 16:57:03.891075 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a" I0128 16:57:03.891195 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-9b7147be-68ba-4eb9-92d4-d5ce63eebc2a successfully I0128 16:57:03.891226 1 utils.go:84] GRPC response: {} I0128 16:58:09.511879 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 16:58:09.511900 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/0c4b64f6f5c2180eba19d7030bdae094b293e0cf4db95e2bc45242685a0bc2c4/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-493b1cf0-3714-47c0-985d-e5673f50f767","csi.storage.k8s.io/pvc/name":"pvc-9v89z","csi.storage.k8s.io/pvc/namespace":"azuredisk-1166","requestedsizegib":"10","skuName":"StandardSSD_ZRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-493b1cf0-3714-47c0-985d-e5673f50f767"} I0128 16:58:11.184222 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 668 lines ... I0128 17:26:14.876698 1 utils.go:84] GRPC response: {} I0128 17:26:14.930456 1 utils.go:77] GRPC call: /csi.v1.Node/NodeUnstageVolume I0128 17:26:14.930478 1 utils.go:78] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf","volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf"} I0128 17:26:14.930550 1 nodeserver.go:201] NodeUnstageVolume: unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf I0128 17:26:14.930582 1 mount_helper_common.go:93] unmounting "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf" (corruptedMount: false, mounterCanSkipMountPointChecks: true) I0128 17:26:14.930596 1 mount_linux.go:362] Unmounting /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf I0128 17:26:14.931876 1 mount_linux.go:375] ignoring 'not mounted' error for /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf I0128 17:26:14.931894 1 mount_helper_common.go:150] Warning: deleting path "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf" I0128 17:26:14.932102 1 nodeserver.go:206] NodeUnstageVolume: unmount /var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-de94b3c1-4134-46ec-a558-f91a0d6820cf successfully I0128 17:26:14.932143 1 utils.go:84] GRPC response: {} I0128 17:27:13.891364 1 utils.go:77] GRPC call: /csi.v1.Node/NodeStageVolume I0128 17:27:13.891408 1 utils.go:78] GRPC request: {"publish_context":{"LUN":"0"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/disk.csi.azure.com/06401ea1821274be274f1aab11734a5c80798e6c3fee2c0707743eb1d84ffce4/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":7}},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-e89313a3-9594-43c2-b984-60361588978b","csi.storage.k8s.io/pvc/name":"pvc-8hlfh","csi.storage.k8s.io/pvc/namespace":"azuredisk-3173","device-setting/device/queue_depth":"17","device-setting/queue/max_sectors_kb":"211","device-setting/queue/nr_requests":"44","device-setting/queue/read_ahead_kb":"256","device-setting/queue/rotational":"0","device-setting/queue/scheduler":"none","device-setting/queue/wbt_lat_usec":"0","perfProfile":"advanced","requestedsizegib":"10","skuname":"StandardSSD_LRS","storage.kubernetes.io/csiProvisionerIdentity":"1674922993979-8081-disk.csi.azure.com"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/capz-v4slna/providers/Microsoft.Compute/disks/pvc-e89313a3-9594-43c2-b984-60361588978b"} I0128 17:27:15.473349 1 azure_common_linux.go:185] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/ ... skipping 648 lines ... cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="300"} 59 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="600"} 59 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="1200"} 59 cloudprovider_azure_op_duration_seconds_bucket{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e",le="+Inf"} 59 cloudprovider_azure_op_duration_seconds_sum{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 826.6198131049999 cloudprovider_azure_op_duration_seconds_count{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 59 # HELP cloudprovider_azure_op_failure_count [ALPHA] Number of failed Azure service operations # TYPE cloudprovider_azure_op_failure_count counter cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_delete_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 2 cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_expand_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 1 cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_publish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 1 cloudprovider_azure_op_failure_count{request="azuredisk_csi_driver_controller_unpublish_volume",resource_group="capz-v4slna",source="disk.csi.azure.com",subscription_id="0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e"} 1 # HELP disabled_metric_total [ALPHA] The count of disabled metrics. ... skipping 70 lines ... # HELP go_gc_heap_objects_objects Number of objects, live or unswept, occupying heap memory. # TYPE go_gc_heap_objects_objects gauge go_gc_heap_objects_objects 31657 # HELP go_gc_heap_tiny_allocs_objects_total Count of small allocations that are packed together into blocks. These allocations are counted separately from other allocations because each individual allocation is not tracked by the runtime, only their block. Each block is already accounted for in allocs-by-size and frees-by-size. # TYPE go_gc_heap_tiny_allocs_objects_total counter go_gc_heap_tiny_allocs_objects_total 38871 # HELP go_gc_limiter_last_enabled_gc_cycle GC cycle the last time the GC CPU limiter was enabled. This metric is useful for diagnosing the root cause of an out-of-memory error, because the limiter trades memory for CPU time when the GC's CPU time gets too high. This is most likely to occur with use of SetMemoryLimit. The first GC cycle is cycle 1, so a value of 0 indicates that it was never enabled. # TYPE go_gc_limiter_last_enabled_gc_cycle gauge go_gc_limiter_last_enabled_gc_cycle 0 # HELP go_gc_pauses_seconds Distribution individual GC-related stop-the-world pause latencies. # TYPE go_gc_pauses_seconds histogram go_gc_pauses_seconds_bucket{le="9.999999999999999e-10"} 0 go_gc_pauses_seconds_bucket{le="9.999999999999999e-09"} 0 ... skipping 259 lines ... # HELP go_gc_heap_objects_objects Number of objects, live or unswept, occupying heap memory. # TYPE go_gc_heap_objects_objects gauge go_gc_heap_objects_objects 16923 # HELP go_gc_heap_tiny_allocs_objects_total Count of small allocations that are packed together into blocks. These allocations are counted separately from other allocations because each individual allocation is not tracked by the runtime, only their block. Each block is already accounted for in allocs-by-size and frees-by-size. # TYPE go_gc_heap_tiny_allocs_objects_total counter go_gc_heap_tiny_allocs_objects_total 3801 # HELP go_gc_limiter_last_enabled_gc_cycle GC cycle the last time the GC CPU limiter was enabled. This metric is useful for diagnosing the root cause of an out-of-memory error, because the limiter trades memory for CPU time when the GC's CPU time gets too high. This is most likely to occur with use of SetMemoryLimit. The first GC cycle is cycle 1, so a value of 0 indicates that it was never enabled. # TYPE go_gc_limiter_last_enabled_gc_cycle gauge go_gc_limiter_last_enabled_gc_cycle 0 # HELP go_gc_pauses_seconds Distribution individual GC-related stop-the-world pause latencies. # TYPE go_gc_pauses_seconds histogram go_gc_pauses_seconds_bucket{le="9.999999999999999e-10"} 0 go_gc_pauses_seconds_bucket{le="9.999999999999999e-09"} 0 ... skipping 274 lines ... [AfterSuite] [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/suite_test.go:165[0m [38;5;243m------------------------------[0m [38;5;9m[1mSummarizing 1 Failure:[0m [38;5;9m[FAIL][0m [0mDynamic Provisioning [38;5;243m[single-az] [0m[38;5;9m[1m[It] should clone a volume of larger size than the source volume and make sure the filesystem is appropriately adjusted [disk.csi.azure.com][0m[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:823[0m [38;5;9m[1mRan 36 of 66 Specs in 4481.639 seconds[0m [38;5;9m[1mFAIL![0m -- [38;5;10m[1m35 Passed[0m | [38;5;9m[1m1 Failed[0m | [38;5;11m[1m0 Pending[0m | [38;5;14m[1m30 Skipped[0m [38;5;228mYou're using deprecated Ginkgo functionality:[0m [38;5;228m=============================================[0m [38;5;11mSupport for custom reporters has been removed in V2. Please read the documentation linked to below for Ginkgo's new behavior and for a migration path:[0m [1mLearn more at:[0m [38;5;14m[4mhttps://onsi.github.io/ginkgo/MIGRATING_TO_V2#removed-custom-reporters[0m [38;5;243mTo silence deprecations that can be silenced set the following environment variable:[0m [38;5;243mACK_GINKGO_DEPRECATIONS=2.4.0[0m --- FAIL: TestE2E (4481.64s) FAIL FAIL sigs.k8s.io/azuredisk-csi-driver/test/e2e 4481.717s FAIL make: *** [Makefile:261: e2e-test] Error 1 NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME capz-v4slna-control-plane-jxnw2 Ready control-plane 79m v1.24.6 10.0.0.4 <none> Ubuntu 20.04.5 LTS 5.15.0-1020-azure containerd://1.6.2 capz-v4slna-md-0-bq7hk Ready <none> 77m v1.24.6 10.1.0.4 <none> Ubuntu 20.04.5 LTS 5.15.0-1020-azure containerd://1.6.2 capz-v4slna-md-0-s8npw Ready <none> 77m v1.24.6 10.1.0.5 <none> Ubuntu 20.04.5 LTS 5.15.0-1020-azure containerd://1.6.2 NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES calico-apiserver calico-apiserver-5ddf677bbb-jqtd5 1/1 Running 0 77m 192.168.37.71 capz-v4slna-control-plane-jxnw2 <none> <none> ... skipping 89 lines ... Jan 28 17:33:59.864: INFO: Describing Pod kube-system/kube-scheduler-capz-v4slna-control-plane-jxnw2 Jan 28 17:33:59.864: INFO: Creating log watcher for controller kube-system/kube-scheduler-capz-v4slna-control-plane-jxnw2, container kube-scheduler Jan 28 17:34:00.262: INFO: Fetching kube-system pod logs took 8.586731143s Jan 28 17:34:00.262: INFO: Dumping workload cluster default/capz-v4slna Azure activity log Jan 28 17:34:00.262: INFO: Creating log watcher for controller tigera-operator/tigera-operator-65d6bf4d4f-xdxpm, container tigera-operator Jan 28 17:34:00.262: INFO: Describing Pod tigera-operator/tigera-operator-65d6bf4d4f-xdxpm Jan 28 17:34:07.813: INFO: Got error while iterating over activity logs for resource group capz-v4slna: insights.ActivityLogsClient#listNextResults: Failure responding to next results request: StatusCode=404 -- Original Error: autorest/azure: error response cannot be parsed: {"<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n<head>\r\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=iso-8859-1\"/>\r\n<title>404 - File or directory not found.</title>\r\n<style type=\"text/css\">\r\n<!--\r\nbody{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}\r\nfieldset{padding:0 15px 10px 15px;} \r\nh1{font-size:2.4em;margin:0;color:#FFF;}\r\nh2{font-si" '\x00' '\x00'} error: invalid character '<' looking for beginning of value Jan 28 17:34:07.814: INFO: Fetching activity logs took 7.551209853s ================ REDACTING LOGS ================ All sensitive variables are redacted cluster.cluster.x-k8s.io "capz-v4slna" deleted /home/prow/go/src/sigs.k8s.io/cluster-api-provider-azure/hack/tools/bin/kind-v0.17.0 delete cluster --name=capz || true Deleting cluster "capz" ... ... skipping 13 lines ...