This job view page is being replaced by Spyglass soon. Check out the new job view.
PRsunpa93: [V2] feat: update volume attachment with publish context upon attachment completion
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2022-06-11 00:56
Elapsed3h15m
Revisionc45bf44865cb97f31953bd75dc41209e9bc3ebe6
Refs 1361

No Test Failures!


Error lines from build-log.txt

... skipping 85 lines ...

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 11156  100 11156    0     0  76410      0 --:--:-- --:--:-- --:--:-- 76937
Downloading https://get.helm.sh/helm-v3.9.0-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
docker pull k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78 || make container-all push-manifest
Error response from daemon: manifest for k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78 not found: manifest unknown: manifest tagged by "latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78" is not found
make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=029b76ce24cfceed8d37e3ee31c228cd7ed57a78 -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2022-06-11T01:04:07Z -extldflags "-static"" -tags azurediskv2 -mod vendor -o _output/amd64/azurediskpluginv2.exe ./pkg/azurediskplugin
docker buildx rm container-builder || true
error: no builder "container-builder" found
docker buildx create --use --name=container-builder
container-builder
# enable qemu for arm64 build
# https://github.com/docker/buildx/issues/464#issuecomment-741507760
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
Unable to find image 'tonistiigi/binfmt:latest' locally
... skipping 688 lines ...
         }
      }
   ]
}
make[2]: Leaving directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
docker pull k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78 || make azdiskschedulerextender-all push-manifest-azdiskschedulerextender
Error response from daemon: manifest for k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78 not found: manifest unknown: manifest tagged by "latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78" is not found
make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
docker buildx rm container-builder || true
docker buildx create --use --name=container-builder
container-builder
# enable qemu for arm64 build
# https://github.com/docker/buildx/issues/464#issuecomment-741507760
... skipping 867 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 60 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 254 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 108 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 244 lines ...
            - volumeName
            - volume_context
            - volume_id
            type: object
          status:
            description: status represents the current state of AzVolumeAttachment.
              includes error, state, and attachment status
            properties:
              annotation:
                additionalProperties:
                  type: string
                description: Annotations contains additional resource information
                  to guide driver actions
... skipping 13 lines ...
                  role:
                    description: The current attachment role.
                    type: string
                required:
                - role
                type: object
              error:
                description: Error occurred during attach/detach of volume
                properties:
                  code:
                    type: string
                  message:
                    type: string
                  parameters:
... skipping 173 lines ...
            - maxMountReplicaCount
            - volumeCapability
            - volumeName
            type: object
          status:
            description: status represents the current state of AzVolume. includes
              error, state, and volume status
            properties:
              annotation:
                additionalProperties:
                  type: string
                description: Annotations contains additional resource information
                  to guide driver actions
... skipping 34 lines ...
                    type: string
                required:
                - capacity_bytes
                - node_expansion_required
                - volume_id
                type: object
              error:
                description: Error occurred during creation/deletion of volume
                properties:
                  code:
                    type: string
                  message:
                    type: string
                  parameters:
... skipping 1061 lines ...
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.4.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - "--leader-election-namespace=kube-system"
            - '-handle-volume-inuse-error=false'
            - '-feature-gates=RecoverVolumeExpansionFailure=true'
            - "-timeout=240s"
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
... skipping 429 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Jun 11 01:12:43.247: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-29z4x" in namespace "azuredisk-7552" to be "Succeeded or Failed"
Jun 11 01:12:43.349: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 102.126139ms
Jun 11 01:12:45.453: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 2.206180374s
Jun 11 01:12:47.557: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 4.309715844s
Jun 11 01:12:49.663: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 6.415594683s
Jun 11 01:12:51.766: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 8.519266233s
Jun 11 01:12:53.871: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 10.624296729s
... skipping 417 lines ...
Jun 11 01:27:34.171: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.92407119s
Jun 11 01:27:36.275: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.027594183s
Jun 11 01:27:38.379: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.131986669s
Jun 11 01:27:40.483: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.235821198s
Jun 11 01:27:42.588: INFO: Pod "azuredisk-volume-tester-29z4x": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.341439884s
Jun 11 01:27:44.589: INFO: deleting Pod "azuredisk-7552"/"azuredisk-volume-tester-29z4x"
Jun 11 01:27:44.734: INFO: Error getting logs for pod azuredisk-volume-tester-29z4x: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-29z4x)
STEP: Deleting pod azuredisk-volume-tester-29z4x in namespace azuredisk-7552
STEP: validating provisioned PV
STEP: checking the PV
Jun 11 01:27:45.045: INFO: deleting PVC "azuredisk-7552"/"pvc-bh8qn"
Jun 11 01:27:45.045: INFO: Deleting PersistentVolumeClaim "pvc-bh8qn"
STEP: waiting for claim's PV "pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad" to be deleted
... skipping 123 lines ...
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:12:43 +0000 UTC - event for pvc-bh8qn: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-7552/pvc-bh8qn"
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:12:43 +0000 UTC - event for pvc-bh8qn: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:12:43 +0000 UTC - event for pvc-bh8qn: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:12:45 +0000 UTC - event for pvc-bh8qn: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:12:58 +0000 UTC - event for azuredisk-volume-tester-29z4x: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad" 
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:14:49 +0000 UTC - event for azuredisk-volume-tester-29z4x: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[kube-api-access-2lxlj test-volume-1]: timed out waiting for the condition
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:14:59 +0000 UTC - event for azuredisk-volume-tester-29z4x: {kubelet k8s-agentpool1-27693060-0} FailedMount: MountVolume.MountDevice failed for volume "pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:19:21 +0000 UTC - event for azuredisk-volume-tester-29z4x: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-2lxlj]: timed out waiting for the condition
Jun 11 01:37:47.730: INFO: At 2022-06-11 01:25:14 +0000 UTC - event for azuredisk-volume-tester-29z4x: {kubelet k8s-agentpool1-27693060-0} FailedMount: MountVolume.MountDevice failed for volume "pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad) to be attached to node (k8s-agentpool1-27693060-0): context deadline exceeded
Jun 11 01:37:47.831: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Jun 11 01:37:47.831: INFO: 
Jun 11 01:37:48.034: INFO: 
Logging node info for node k8s-agentpool1-27693060-0
Jun 11 01:37:48.137: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-27693060-0    3da95b76-1ec0-48f3-be97-0f3b51c5ea22 7192 0 2022-06-11 01:00:14 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:northeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vemmrhlo kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-27693060-0 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone: topology.kubernetes.io/region:northeurope topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool1-27693060-0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-06-11 01:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-06-11 01:00:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-06-11 01:00:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kubelet Update v1 2022-06-11 01:30:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2022-06-11 01:36:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-27693060-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7287767040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6501335040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-06-11 01:00:48 +0000 UTC,LastTransitionTime:2022-06-11 01:00:48 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-06-11 01:35:21 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-06-11 01:35:21 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-06-11 01:35:21 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-06-11 01:35:21 +0000 UTC,LastTransitionTime:2022-06-11 01:00:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-agentpool1-27693060-0,},NodeAddress{Type:InternalIP,Address:10.240.0.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:35e09b798003485181979a7a3c9d2b38,SystemUUID:aaf73a93-cb53-4541-9615-c3f03cff2aa3,BootID:c10a22ca-21f4-4fde-b65b-eec5a1fdd392,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.23.7,KubeProxyVersion:v1.23.7,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:bb0ff1bc40f2c5c080c5f56fd9919055fd532e33458b290b96dc7c6996a65121 k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:136448494,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:0c890e918d42c158fab68867bade2ab1db04c3408452db9611a264ba3e43e6b3 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.7],SizeBytes:112296063,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[k8sprow.azurecr.io/azdiskschedulerextender-csi@sha256:ba7aa30ee814ce9003985ee9b5c26ca14f561920df709bef7f41830eba6bf991 k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:56759982,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:c35f45e8dfaa30363e589296415a057f9d32813920fc3820d6557491657f199a mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.2],SizeBytes:50607491,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Jun 11 01:37:48.138: INFO: 
... skipping 90 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44
    should create a volume on demand with mount options [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [csi-azuredisk-scheduler-extender] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:102

    Unexpected error:
        <*errors.errorString | 0xc0004504a0>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-29z4x\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-29z4x" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/resources/pod.go:89
------------------------------
Dynamic Provisioning [single-az] 
  should create a pod with volume mount subpath [disk.csi.azure.com] [Windows]
... skipping 97 lines ...
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod has 'FailedMount' event
Jun 11 01:40:13.827: INFO: deleting Pod "azuredisk-7640"/"azuredisk-volume-tester-c2hq6"
Jun 11 01:40:13.952: INFO: Error getting logs for pod azuredisk-volume-tester-c2hq6: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-c2hq6)
STEP: Deleting pod azuredisk-volume-tester-c2hq6 in namespace azuredisk-7640
STEP: validating provisioned PV
STEP: checking the PV
Jun 11 01:40:14.260: INFO: deleting PVC "azuredisk-7640"/"pvc-stqnv"
Jun 11 01:40:14.260: INFO: Deleting PersistentVolumeClaim "pvc-stqnv"
STEP: waiting for claim's PV "pvc-3245832d-89c3-4140-bddd-67308ecc97da" to be deleted
... skipping 123 lines ...
Jun 11 01:50:16.944: INFO: At 2022-06-11 01:37:57 +0000 UTC - event for pvc-stqnv: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jun 11 01:50:16.944: INFO: At 2022-06-11 01:37:57 +0000 UTC - event for pvc-stqnv: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Jun 11 01:50:16.944: INFO: At 2022-06-11 01:37:57 +0000 UTC - event for pvc-stqnv: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-7640/pvc-stqnv"
Jun 11 01:50:16.944: INFO: At 2022-06-11 01:38:00 +0000 UTC - event for pvc-stqnv: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-3245832d-89c3-4140-bddd-67308ecc97da
Jun 11 01:50:16.944: INFO: At 2022-06-11 01:38:13 +0000 UTC - event for azuredisk-volume-tester-c2hq6: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-3245832d-89c3-4140-bddd-67308ecc97da" 
Jun 11 01:50:16.944: INFO: At 2022-06-11 01:40:03 +0000 UTC - event for azuredisk-volume-tester-c2hq6: {kubelet k8s-agentpool1-27693060-1} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-kjdk4]: timed out waiting for the condition
Jun 11 01:50:16.944: INFO: At 2022-06-11 01:40:13 +0000 UTC - event for azuredisk-volume-tester-c2hq6: {kubelet k8s-agentpool1-27693060-1} FailedMount: MountVolume.MountDevice failed for volume "pvc-3245832d-89c3-4140-bddd-67308ecc97da" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 01:50:17.045: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Jun 11 01:50:17.045: INFO: 
Jun 11 01:50:17.247: INFO: 
Logging node info for node k8s-agentpool1-27693060-0
Jun 11 01:50:17.350: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-27693060-0    3da95b76-1ec0-48f3-be97-0f3b51c5ea22 9885 0 2022-06-11 01:00:14 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:northeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vemmrhlo kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-27693060-0 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone: topology.kubernetes.io/region:northeurope topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool1-27693060-0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-06-11 01:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-06-11 01:00:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-06-11 01:00:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kubelet Update v1 2022-06-11 01:30:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2022-06-11 01:48:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-27693060-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7287767040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6501335040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-06-11 01:00:48 +0000 UTC,LastTransitionTime:2022-06-11 01:00:48 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-06-11 01:45:34 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-06-11 01:45:34 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-06-11 01:45:34 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-06-11 01:45:34 +0000 UTC,LastTransitionTime:2022-06-11 01:00:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-agentpool1-27693060-0,},NodeAddress{Type:InternalIP,Address:10.240.0.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:35e09b798003485181979a7a3c9d2b38,SystemUUID:aaf73a93-cb53-4541-9615-c3f03cff2aa3,BootID:c10a22ca-21f4-4fde-b65b-eec5a1fdd392,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.23.7,KubeProxyVersion:v1.23.7,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:bb0ff1bc40f2c5c080c5f56fd9919055fd532e33458b290b96dc7c6996a65121 k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:136448494,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:0c890e918d42c158fab68867bade2ab1db04c3408452db9611a264ba3e43e6b3 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.7],SizeBytes:112296063,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[k8sprow.azurecr.io/azdiskschedulerextender-csi@sha256:ba7aa30ee814ce9003985ee9b5c26ca14f561920df709bef7f41830eba6bf991 k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:56759982,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:c35f45e8dfaa30363e589296415a057f9d32813920fc3820d6557491657f199a mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.2],SizeBytes:50607491,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad,DevicePath:,},},Config:nil,},}
Jun 11 01:50:17.351: INFO: 
... skipping 90 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44
    should receive FailedMount event with invalid mount options [kubernetes.io/azure-disk] [disk.csi.azure.com] [csi-azuredisk-scheduler-extender] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:269

    Unexpected error:
        <*errors.errorString | 0xc000cc41f0>: {
            s: "PersistentVolume pvc-3245832d-89c3-4140-bddd-67308ecc97da still exists within 10m0s",
        }
        PersistentVolume pvc-3245832d-89c3-4140-bddd-67308ecc97da still exists within 10m0s
    occurred

... skipping 15 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Jun 11 01:50:21.452: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-ncwv9" in namespace "azuredisk-2925" to be "Succeeded or Failed"
Jun 11 01:50:21.554: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 102.244571ms
Jun 11 01:50:23.657: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.20517s
Jun 11 01:50:25.761: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.309654967s
Jun 11 01:50:27.866: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.413965342s
Jun 11 01:50:29.969: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.5170031s
Jun 11 01:50:32.072: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.620108741s
... skipping 417 lines ...
Jun 11 02:05:11.353: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m49.90108341s
Jun 11 02:05:13.456: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.004434734s
Jun 11 02:05:15.559: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.107262268s
Jun 11 02:05:17.662: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.209870704s
Jun 11 02:05:19.766: INFO: Pod "azuredisk-volume-tester-ncwv9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.314059203s
Jun 11 02:05:21.767: INFO: deleting Pod "azuredisk-2925"/"azuredisk-volume-tester-ncwv9"
Jun 11 02:05:21.913: INFO: Error getting logs for pod azuredisk-volume-tester-ncwv9: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-ncwv9)
STEP: Deleting pod azuredisk-volume-tester-ncwv9 in namespace azuredisk-2925
STEP: validating provisioned PV
STEP: checking the PV
Jun 11 02:05:22.223: INFO: deleting PVC "azuredisk-2925"/"pvc-bxgqx"
Jun 11 02:05:22.223: INFO: Deleting PersistentVolumeClaim "pvc-bxgqx"
STEP: waiting for claim's PV "pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24" to be deleted
... skipping 123 lines ...
Jun 11 02:15:24.915: INFO: At 2022-06-11 01:50:21 +0000 UTC - event for pvc-bxgqx: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-2925/pvc-bxgqx"
Jun 11 02:15:24.915: INFO: At 2022-06-11 01:50:21 +0000 UTC - event for pvc-bxgqx: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Jun 11 02:15:24.915: INFO: At 2022-06-11 01:50:21 +0000 UTC - event for pvc-bxgqx: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jun 11 02:15:24.915: INFO: At 2022-06-11 01:50:24 +0000 UTC - event for pvc-bxgqx: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24
Jun 11 02:15:24.916: INFO: At 2022-06-11 01:50:41 +0000 UTC - event for azuredisk-volume-tester-ncwv9: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24" 
Jun 11 02:15:24.916: INFO: At 2022-06-11 01:52:27 +0000 UTC - event for azuredisk-volume-tester-ncwv9: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[kube-api-access-bw8f7 test-volume-1]: timed out waiting for the condition
Jun 11 02:15:24.916: INFO: At 2022-06-11 01:52:42 +0000 UTC - event for azuredisk-volume-tester-ncwv9: {kubelet k8s-agentpool1-27693060-0} FailedMapVolume: MapVolume.SetUpDevice failed for volume "pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 02:15:24.916: INFO: At 2022-06-11 02:00:49 +0000 UTC - event for azuredisk-volume-tester-ncwv9: {kubelet k8s-agentpool1-27693060-0} FailedMapVolume: MapVolume.SetUpDevice failed for volume "pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24) to be attached to node (k8s-agentpool1-27693060-0): context deadline exceeded
Jun 11 02:15:24.916: INFO: At 2022-06-11 02:03:47 +0000 UTC - event for azuredisk-volume-tester-ncwv9: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-bw8f7]: timed out waiting for the condition
Jun 11 02:15:25.017: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Jun 11 02:15:25.017: INFO: 
Jun 11 02:15:25.219: INFO: 
Logging node info for node k8s-agentpool1-27693060-0
Jun 11 02:15:25.322: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-27693060-0    3da95b76-1ec0-48f3-be97-0f3b51c5ea22 15631 0 2022-06-11 01:00:14 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:northeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vemmrhlo kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-27693060-0 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone: topology.kubernetes.io/region:northeurope topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool1-27693060-0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-06-11 01:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-06-11 01:00:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-06-11 01:00:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kubelet Update v1 2022-06-11 02:07:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2022-06-11 02:15:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-27693060-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7287767040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6501335040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-06-11 01:00:48 +0000 UTC,LastTransitionTime:2022-06-11 01:00:48 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-06-11 02:12:58 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-06-11 02:12:58 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-06-11 02:12:58 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-06-11 02:12:58 +0000 UTC,LastTransitionTime:2022-06-11 01:00:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-agentpool1-27693060-0,},NodeAddress{Type:InternalIP,Address:10.240.0.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:35e09b798003485181979a7a3c9d2b38,SystemUUID:aaf73a93-cb53-4541-9615-c3f03cff2aa3,BootID:c10a22ca-21f4-4fde-b65b-eec5a1fdd392,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.23.7,KubeProxyVersion:v1.23.7,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:bb0ff1bc40f2c5c080c5f56fd9919055fd532e33458b290b96dc7c6996a65121 k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:136448494,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:0c890e918d42c158fab68867bade2ab1db04c3408452db9611a264ba3e43e6b3 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.7],SizeBytes:112296063,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[k8sprow.azurecr.io/azdiskschedulerextender-csi@sha256:ba7aa30ee814ce9003985ee9b5c26ca14f561920df709bef7f41830eba6bf991 k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:56759982,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:c35f45e8dfaa30363e589296415a057f9d32813920fc3820d6557491657f199a mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.2],SizeBytes:50607491,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad,DevicePath:,},},Config:nil,},}
... skipping 91 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44
    should create a raw block volume on demand [kubernetes.io/azure-disk] [disk.csi.azure.com] [csi-azuredisk-scheduler-extender] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:306

    Unexpected error:
        <*errors.errorString | 0xc000a96b70>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-ncwv9\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-ncwv9" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/resources/pod.go:89
------------------------------
Dynamic Provisioning [single-az] 
  should create a volume on demand and mount it as readOnly in a pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [csi-azuredisk-scheduler-extender]
... skipping 11 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with an error
Jun 11 02:15:29.274: INFO: Waiting up to 10m0s for pod "azuredisk-volume-tester-78tt6" in namespace "azuredisk-9466" to be "Error status code"
Jun 11 02:15:29.376: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 102.187009ms
Jun 11 02:15:31.479: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204446729s
Jun 11 02:15:33.582: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.308252629s
Jun 11 02:15:35.686: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.411833483s
Jun 11 02:15:37.789: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.514625908s
Jun 11 02:15:39.892: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.617282451s
... skipping 275 lines ...
Jun 11 02:25:20.605: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 9m51.330856997s
Jun 11 02:25:22.709: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 9m53.435075279s
Jun 11 02:25:24.814: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 9m55.539719374s
Jun 11 02:25:26.918: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 9m57.643707034s
Jun 11 02:25:29.022: INFO: Pod "azuredisk-volume-tester-78tt6": Phase="Pending", Reason="", readiness=false. Elapsed: 9m59.747405583s
Jun 11 02:25:31.022: INFO: deleting Pod "azuredisk-9466"/"azuredisk-volume-tester-78tt6"
Jun 11 02:25:31.161: INFO: Error getting logs for pod azuredisk-volume-tester-78tt6: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-78tt6)
STEP: Deleting pod azuredisk-volume-tester-78tt6 in namespace azuredisk-9466
STEP: validating provisioned PV
STEP: checking the PV
Jun 11 02:25:31.474: INFO: deleting PVC "azuredisk-9466"/"pvc-k6tst"
Jun 11 02:25:31.474: INFO: Deleting PersistentVolumeClaim "pvc-k6tst"
STEP: waiting for claim's PV "pvc-cbd16349-4480-47e6-b9de-36a66ad71c28" to be deleted
... skipping 123 lines ...
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:15:29 +0000 UTC - event for pvc-k6tst: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:15:29 +0000 UTC - event for pvc-k6tst: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:15:29 +0000 UTC - event for pvc-k6tst: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-9466/pvc-k6tst"
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:15:31 +0000 UTC - event for pvc-k6tst: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-cbd16349-4480-47e6-b9de-36a66ad71c28
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:15:44 +0000 UTC - event for azuredisk-volume-tester-78tt6: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-cbd16349-4480-47e6-b9de-36a66ad71c28" 
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:17:35 +0000 UTC - event for azuredisk-volume-tester-78tt6: {kubelet k8s-agentpool1-27693060-1} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-srcl2]: timed out waiting for the condition
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:17:44 +0000 UTC - event for azuredisk-volume-tester-78tt6: {kubelet k8s-agentpool1-27693060-1} FailedMount: MountVolume.MountDevice failed for volume "pvc-cbd16349-4480-47e6-b9de-36a66ad71c28" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 02:35:34.168: INFO: At 2022-06-11 02:19:50 +0000 UTC - event for azuredisk-volume-tester-78tt6: {kubelet k8s-agentpool1-27693060-1} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[kube-api-access-srcl2 test-volume-1]: timed out waiting for the condition
Jun 11 02:35:34.270: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Jun 11 02:35:34.270: INFO: 
Jun 11 02:35:34.473: INFO: 
Logging node info for node k8s-agentpool1-27693060-0
Jun 11 02:35:34.575: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-27693060-0    3da95b76-1ec0-48f3-be97-0f3b51c5ea22 20094 0 2022-06-11 01:00:14 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:northeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vemmrhlo kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-27693060-0 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone: topology.kubernetes.io/region:northeurope topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool1-27693060-0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-06-11 01:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-06-11 01:00:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-06-11 01:00:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kubelet Update v1 2022-06-11 02:07:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {kube-controller-manager Update v1 2022-06-11 02:17:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-27693060-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7287767040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6501335040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-06-11 01:00:48 +0000 UTC,LastTransitionTime:2022-06-11 01:00:48 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-06-11 02:33:25 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-06-11 02:33:25 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-06-11 02:33:25 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-06-11 02:33:25 +0000 UTC,LastTransitionTime:2022-06-11 01:00:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-agentpool1-27693060-0,},NodeAddress{Type:InternalIP,Address:10.240.0.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:35e09b798003485181979a7a3c9d2b38,SystemUUID:aaf73a93-cb53-4541-9615-c3f03cff2aa3,BootID:c10a22ca-21f4-4fde-b65b-eec5a1fdd392,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.23.7,KubeProxyVersion:v1.23.7,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:bb0ff1bc40f2c5c080c5f56fd9919055fd532e33458b290b96dc7c6996a65121 k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:136448494,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:0c890e918d42c158fab68867bade2ab1db04c3408452db9611a264ba3e43e6b3 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.7],SizeBytes:112296063,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[k8sprow.azurecr.io/azdiskschedulerextender-csi@sha256:ba7aa30ee814ce9003985ee9b5c26ca14f561920df709bef7f41830eba6bf991 k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:56759982,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:c35f45e8dfaa30363e589296415a057f9d32813920fc3820d6557491657f199a mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.2],SizeBytes:50607491,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24,DevicePath:,},},Config:nil,},}
... skipping 91 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44
    should create a volume on demand and mount it as readOnly in a pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [csi-azuredisk-scheduler-extender] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:338

    Unexpected error:
        <*errors.errorString | 0xc00050b1f0>: {
            s: "Gave up after waiting 10m0s for pod \"azuredisk-volume-tester-78tt6\" to be \"Error status code\"",
        }
        Gave up after waiting 10m0s for pod "azuredisk-volume-tester-78tt6" to be "Error status code"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/resources/pod.go:114
------------------------------
Dynamic Provisioning [single-az] 
  should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [csi-azuredisk-scheduler-extender]
... skipping 13 lines ...
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod is running
Jun 11 02:50:38.981: INFO: deleting Pod "azuredisk-2529"/"azuredisk-volume-tester-5ptbh"
Jun 11 02:50:39.112: INFO: Error getting logs for pod azuredisk-volume-tester-5ptbh: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-5ptbh)
STEP: Deleting pod azuredisk-volume-tester-5ptbh in namespace azuredisk-2529
STEP: validating provisioned PV
STEP: checking the PV
Jun 11 02:50:39.424: INFO: deleting PVC "azuredisk-2529"/"pvc-crdx7"
Jun 11 02:50:39.424: INFO: Deleting PersistentVolumeClaim "pvc-crdx7"
STEP: waiting for claim's PV "pvc-5f901a80-d63c-493f-bb1e-3fcf7278f04f" to be deleted
... skipping 123 lines ...
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:35:38 +0000 UTC - event for pvc-crdx7: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:35:38 +0000 UTC - event for pvc-crdx7: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:35:38 +0000 UTC - event for pvc-crdx7: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-2529/pvc-crdx7"
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:35:41 +0000 UTC - event for pvc-crdx7: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-5f901a80-d63c-493f-bb1e-3fcf7278f04f
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:35:53 +0000 UTC - event for azuredisk-volume-tester-5ptbh: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-5f901a80-d63c-493f-bb1e-3fcf7278f04f" 
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:37:44 +0000 UTC - event for azuredisk-volume-tester-5ptbh: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[kube-api-access-t5sl4 test-volume-1]: timed out waiting for the condition
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:37:53 +0000 UTC - event for azuredisk-volume-tester-5ptbh: {kubelet k8s-agentpool1-27693060-0} FailedMount: MountVolume.MountDevice failed for volume "pvc-5f901a80-d63c-493f-bb1e-3fcf7278f04f" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 03:00:42.251: INFO: At 2022-06-11 02:40:01 +0000 UTC - event for azuredisk-volume-tester-5ptbh: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-t5sl4]: timed out waiting for the condition
Jun 11 03:00:42.354: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Jun 11 03:00:42.354: INFO: 
Jun 11 03:00:42.558: INFO: 
Logging node info for node k8s-agentpool1-27693060-0
Jun 11 03:00:42.661: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-27693060-0    3da95b76-1ec0-48f3-be97-0f3b51c5ea22 25469 0 2022-06-11 01:00:14 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:northeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vemmrhlo kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-27693060-0 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone: topology.kubernetes.io/region:northeurope topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool1-27693060-0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-06-11 01:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-06-11 01:00:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-06-11 01:00:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-06-11 02:17:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-06-11 02:53:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-27693060-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7287767040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6501335040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-06-11 01:00:48 +0000 UTC,LastTransitionTime:2022-06-11 01:00:48 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-06-11 02:58:09 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-06-11 02:58:09 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-06-11 02:58:09 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-06-11 02:58:09 +0000 UTC,LastTransitionTime:2022-06-11 01:00:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-agentpool1-27693060-0,},NodeAddress{Type:InternalIP,Address:10.240.0.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:35e09b798003485181979a7a3c9d2b38,SystemUUID:aaf73a93-cb53-4541-9615-c3f03cff2aa3,BootID:c10a22ca-21f4-4fde-b65b-eec5a1fdd392,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.23.7,KubeProxyVersion:v1.23.7,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:bb0ff1bc40f2c5c080c5f56fd9919055fd532e33458b290b96dc7c6996a65121 k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:136448494,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:0c890e918d42c158fab68867bade2ab1db04c3408452db9611a264ba3e43e6b3 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.7],SizeBytes:112296063,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[k8sprow.azurecr.io/azdiskschedulerextender-csi@sha256:ba7aa30ee814ce9003985ee9b5c26ca14f561920df709bef7f41830eba6bf991 k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:56759982,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:c35f45e8dfaa30363e589296415a057f9d32813920fc3820d6557491657f199a mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.2],SizeBytes:50607491,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24,DevicePath:,},},Config:nil,},}
... skipping 91 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44
    should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [csi-azuredisk-scheduler-extender] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:372

    Unexpected error:
        <*errors.errorString | 0xc00032b250>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

... skipping 304 lines ...
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:00:46 +0000 UTC - event for pvc-gbzmv: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:00:46 +0000 UTC - event for pvc-gbzmv: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-1065/pvc-gbzmv"
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:00:46 +0000 UTC - event for pvc-gbzmv: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:00:49 +0000 UTC - event for pvc-gbzmv: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-bd7f895d-0413-4508-b08e-6b117badeaaa
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:01:02 +0000 UTC - event for azuredisk-volume-tester-prlpp-766bf779c8-dhdzj: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-bd7f895d-0413-4508-b08e-6b117badeaaa" 
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:02:52 +0000 UTC - event for azuredisk-volume-tester-prlpp-766bf779c8-dhdzj: {kubelet k8s-agentpool1-27693060-1} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-mc92b]: timed out waiting for the condition
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:03:02 +0000 UTC - event for azuredisk-volume-tester-prlpp-766bf779c8-dhdzj: {kubelet k8s-agentpool1-27693060-1} FailedMount: MountVolume.MountDevice failed for volume "pvc-bd7f895d-0413-4508-b08e-6b117badeaaa" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:05:03 +0000 UTC - event for azuredisk-volume-tester-prlpp-766bf779c8-dhdzj: {kubelet k8s-agentpool1-27693060-1} FailedMount: MountVolume.MountDevice failed for volume "pvc-bd7f895d-0413-4508-b08e-6b117badeaaa" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-bd7f895d-0413-4508-b08e-6b117badeaaa) to be attached to node (k8s-agentpool1-27693060-1): context deadline exceeded
Jun 11 03:15:50.197: INFO: At 2022-06-11 03:07:28 +0000 UTC - event for azuredisk-volume-tester-prlpp-766bf779c8-dhdzj: {kubelet k8s-agentpool1-27693060-1} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[kube-api-access-mc92b test-volume-1]: timed out waiting for the condition
Jun 11 03:15:50.300: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Jun 11 03:15:50.300: INFO: 
Jun 11 03:15:50.502: INFO: 
Logging node info for node k8s-agentpool1-27693060-0
Jun 11 03:15:50.605: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-27693060-0    3da95b76-1ec0-48f3-be97-0f3b51c5ea22 28909 0 2022-06-11 01:00:14 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:northeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vemmrhlo kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-27693060-0 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone: topology.kubernetes.io/region:northeurope topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool1-27693060-0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-06-11 01:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-06-11 01:00:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-06-11 01:00:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-06-11 02:17:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-06-11 02:53:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-27693060-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7287767040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6501335040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-06-11 01:00:48 +0000 UTC,LastTransitionTime:2022-06-11 01:00:48 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-06-11 03:13:28 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-06-11 03:13:28 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-06-11 03:13:28 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-06-11 03:13:28 +0000 UTC,LastTransitionTime:2022-06-11 01:00:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-agentpool1-27693060-0,},NodeAddress{Type:InternalIP,Address:10.240.0.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:35e09b798003485181979a7a3c9d2b38,SystemUUID:aaf73a93-cb53-4541-9615-c3f03cff2aa3,BootID:c10a22ca-21f4-4fde-b65b-eec5a1fdd392,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.23.7,KubeProxyVersion:v1.23.7,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:bb0ff1bc40f2c5c080c5f56fd9919055fd532e33458b290b96dc7c6996a65121 k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:136448494,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:0c890e918d42c158fab68867bade2ab1db04c3408452db9611a264ba3e43e6b3 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.7],SizeBytes:112296063,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[k8sprow.azurecr.io/azdiskschedulerextender-csi@sha256:ba7aa30ee814ce9003985ee9b5c26ca14f561920df709bef7f41830eba6bf991 k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:56759982,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:c35f45e8dfaa30363e589296415a057f9d32813920fc3820d6557491657f199a mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.2],SizeBytes:50607491,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-5f901a80-d63c-493f-bb1e-3fcf7278f04f,DevicePath:,},},Config:nil,},}
... skipping 91 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44
    should create a deployment object, write and read to it, delete the pod and write and read to it again [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [csi-azuredisk-scheduler-extender] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:436

    Unexpected error:
        <*errors.errorString | 0xc000451b00>: {
            s: "error waiting for deployment \"azuredisk-volume-tester-prlpp\" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:\"Available\", Status:\"False\", LastUpdateTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), LastTransitionTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), Reason:\"MinimumReplicasUnavailable\", Message:\"Deployment does not have minimum availability.\"}, v1.DeploymentCondition{Type:\"Progressing\", Status:\"True\", LastUpdateTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), LastTransitionTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), Reason:\"ReplicaSetUpdated\", Message:\"ReplicaSet \\\"azuredisk-volume-tester-prlpp-766bf779c8\\\" is progressing.\"}}, CollisionCount:(*int32)(nil)}",
        }
        error waiting for deployment "azuredisk-volume-tester-prlpp" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), LastTransitionTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), LastTransitionTime:time.Date(2022, time.June, 11, 3, 0, 46, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"azuredisk-volume-tester-prlpp-766bf779c8\" is progressing."}}, CollisionCount:(*int32)(nil)}
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/resources/deployment.go:126
------------------------------
Dynamic Provisioning [single-az] 
  should delete PV with reclaimPolicy "Delete" [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows]
... skipping 150 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Jun 11 03:16:17.859: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fhpcs" in namespace "azuredisk-7422" to be "Succeeded or Failed"
Jun 11 03:16:17.960: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 101.6856ms
Jun 11 03:16:20.063: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 2.204112403s
Jun 11 03:16:22.167: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 4.308659645s
Jun 11 03:16:24.270: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 6.41084446s
Jun 11 03:16:26.373: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 8.513920822s
Jun 11 03:16:28.476: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 10.617578097s
... skipping 417 lines ...
Jun 11 03:31:07.845: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 14m49.985925254s
Jun 11 03:31:09.948: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.089295504s
Jun 11 03:31:12.050: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.191549937s
Jun 11 03:31:14.153: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.294106673s
Jun 11 03:31:16.257: INFO: Pod "azuredisk-volume-tester-fhpcs": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.397740411s
Jun 11 03:31:18.257: INFO: deleting Pod "azuredisk-7422"/"azuredisk-volume-tester-fhpcs"
Jun 11 03:31:18.394: INFO: Error getting logs for pod azuredisk-volume-tester-fhpcs: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-fhpcs)
STEP: Deleting pod azuredisk-volume-tester-fhpcs in namespace azuredisk-7422
STEP: validating provisioned PV
STEP: checking the PV
Jun 11 03:31:18.705: INFO: deleting PVC "azuredisk-7422"/"pvc-kq45n"
Jun 11 03:31:18.705: INFO: Deleting PersistentVolumeClaim "pvc-kq45n"
STEP: waiting for claim's PV "pvc-162daa04-7b0e-4175-9851-a743a5da8af4" to be deleted
... skipping 311 lines ...
Jun 11 03:56:31.096: INFO: PersistentVolume pvc-a7684656-473d-492c-bb97-43cd5b6052f7 found and phase=Released (5m6.424762123s)
Jun 11 03:56:36.201: INFO: PersistentVolume pvc-a7684656-473d-492c-bb97-43cd5b6052f7 found and phase=Released (5m11.529255911s)
Jun 11 03:56:41.304: INFO: PersistentVolume pvc-a7684656-473d-492c-bb97-43cd5b6052f7 found and phase=Released (5m16.632512744s)
Jun 11 03:56:46.409: INFO: PersistentVolume pvc-a7684656-473d-492c-bb97-43cd5b6052f7 found and phase=Released (5m21.738102169s)
Jun 11 03:56:51.512: INFO: PersistentVolume pvc-a7684656-473d-492c-bb97-43cd5b6052f7 found and phase=Released (5m26.84041524s)
Jun 11 03:56:56.618: INFO: PersistentVolume pvc-a7684656-473d-492c-bb97-43cd5b6052f7 found and phase=Released (5m31.946469169s)
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:165","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 3h0m0s timeout","severity":"error","time":"2022-06-11T03:57:01Z"}
++ early_exit_handler
++ '[' -n 177 ']'
++ kill -TERM 177
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 82 lines ...
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:16:20 +0000 UTC - event for pvc-kq45n: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-162daa04-7b0e-4175-9851-a743a5da8af4
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:16:20 +0000 UTC - event for pvc-q26z5: {disk.csi.azure.com_k8s-agentpool1-27693060-1_c39a5e5f-a951-408b-9b2a-fd158a269791 } ProvisioningSucceeded: Successfully provisioned volume pvc-e7579a8a-6b22-4aad-8b7c-04a9b77afeb1
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:16:33 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-e7579a8a-6b22-4aad-8b7c-04a9b77afeb1" 
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:16:33 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-a7684656-473d-492c-bb97-43cd5b6052f7" 
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:16:33 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-162daa04-7b0e-4175-9851-a743a5da8af4" 
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:18:23 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1 test-volume-2 test-volume-3], unattached volumes=[test-volume-1 test-volume-2 test-volume-3 kube-api-access-xrrc5]: timed out waiting for the condition
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:18:33 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {kubelet k8s-agentpool1-27693060-0} FailedMount: MountVolume.MountDevice failed for volume "pvc-e7579a8a-6b22-4aad-8b7c-04a9b77afeb1" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:18:33 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {kubelet k8s-agentpool1-27693060-0} FailedMount: MountVolume.MountDevice failed for volume "pvc-a7684656-473d-492c-bb97-43cd5b6052f7" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:18:33 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {kubelet k8s-agentpool1-27693060-0} FailedMount: MountVolume.MountDevice failed for volume "pvc-162daa04-7b0e-4175-9851-a743a5da8af4" : rpc error: code = DeadlineExceeded desc = context deadline exceeded
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:20:41 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-2 test-volume-3 test-volume-1], unattached volumes=[test-volume-2 test-volume-3 kube-api-access-xrrc5 test-volume-1]: timed out waiting for the condition
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:24:37 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {kubelet k8s-agentpool1-27693060-0} FailedMount: MountVolume.MountDevice failed for volume "pvc-162daa04-7b0e-4175-9851-a743a5da8af4" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-162daa04-7b0e-4175-9851-a743a5da8af4) to be attached to node (k8s-agentpool1-27693060-0): context deadline exceeded
Jun 11 04:01:27.453: INFO: At 2022-06-11 03:27:29 +0000 UTC - event for azuredisk-volume-tester-fhpcs: {kubelet k8s-agentpool1-27693060-0} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-3 test-volume-1 test-volume-2], unattached volumes=[test-volume-3 kube-api-access-xrrc5 test-volume-1 test-volume-2]: timed out waiting for the condition
Jun 11 04:01:27.556: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Jun 11 04:01:27.556: INFO: 
Jun 11 04:01:27.761: INFO: 
Logging node info for node k8s-agentpool1-27693060-0
Jun 11 04:01:27.865: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-27693060-0    3da95b76-1ec0-48f3-be97-0f3b51c5ea22 39268 0 2022-06-11 01:00:14 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:northeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vemmrhlo kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-27693060-0 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone: topology.kubernetes.io/region:northeurope topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool1-27693060-0"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-06-11 01:00:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-06-11 01:00:19 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-06-11 01:00:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-06-11 02:17:03 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-06-11 03:33:44 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/virtualMachines/k8s-agentpool1-27693060-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7287767040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{8 0} {<nil>} 8 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6501335040 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-06-11 01:00:48 +0000 UTC,LastTransitionTime:2022-06-11 01:00:48 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-06-11 03:59:17 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-06-11 03:59:17 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-06-11 03:59:17 +0000 UTC,LastTransitionTime:2022-06-11 01:00:12 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-06-11 03:59:17 +0000 UTC,LastTransitionTime:2022-06-11 01:00:28 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-agentpool1-27693060-0,},NodeAddress{Type:InternalIP,Address:10.240.0.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:35e09b798003485181979a7a3c9d2b38,SystemUUID:aaf73a93-cb53-4541-9615-c3f03cff2aa3,BootID:c10a22ca-21f4-4fde-b65b-eec5a1fdd392,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.23.7,KubeProxyVersion:v1.23.7,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:bb0ff1bc40f2c5c080c5f56fd9919055fd532e33458b290b96dc7c6996a65121 k8sprow.azurecr.io/azuredisk-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:136448494,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:0c890e918d42c158fab68867bade2ab1db04c3408452db9611a264ba3e43e6b3 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.7],SizeBytes:112296063,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[k8sprow.azurecr.io/azdiskschedulerextender-csi@sha256:ba7aa30ee814ce9003985ee9b5c26ca14f561920df709bef7f41830eba6bf991 k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-029b76ce24cfceed8d37e3ee31c228cd7ed57a78],SizeBytes:56759982,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:c35f45e8dfaa30363e589296415a057f9d32813920fc3820d6557491657f199a mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.2],SizeBytes:50607491,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-162daa04-7b0e-4175-9851-a743a5da8af4,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e7579a8a-6b22-4aad-8b7c-04a9b77afeb1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-a7684656-473d-492c-bb97-43cd5b6052f7,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-e08e4513-25cc-4423-9ec4-6bf6393e6dad,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-7a172c8e-f805-40f5-92ca-c04a228e5c24,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vemmrhlo/providers/Microsoft.Compute/disks/pvc-5f901a80-d63c-493f-bb1e-3fcf7278f04f,DevicePath:,},},Config:nil,},}
... skipping 91 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:44
    should create multiple PV objects, bind to PVCs and attach all to a single pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [csi-azuredisk-scheduler-extender] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:589

    Unexpected error:
        <*errors.errorString | 0xc0006d0730>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-fhpcs\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-fhpcs" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/resources/pod.go:89
------------------------------
Dynamic Provisioning [single-az] 
  should create a raw block volume and a filesystem volume on demand and bind to the same pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [csi-azuredisk-scheduler-extender]
... skipping 16 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Jun 11 04:01:32.095: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-q8trf" in namespace "azuredisk-4705" to be "Succeeded or Failed"
Jun 11 04:01:32.198: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 103.052862ms
Jun 11 04:01:34.303: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.208142563s
Jun 11 04:01:36.407: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.311943567s
Jun 11 04:01:38.512: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 6.417245973s
Jun 11 04:01:40.617: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 8.521874339s
Jun 11 04:01:42.720: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 10.625179281s
... skipping 287 lines ...
Jun 11 04:11:48.850: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 10m16.754701216s
Jun 11 04:11:50.954: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 10m18.859142038s
Jun 11 04:11:53.058: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 10m20.963558838s
Jun 11 04:11:55.162: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 10m23.067263782s
Jun 11 04:11:57.268: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 10m25.17282837s
Jun 11 04:11:59.372: INFO: Pod "azuredisk-volume-tester-q8trf": Phase="Pending", Reason="", readiness=false. Elapsed: 10m27.277434811s
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:255","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15m0s grace period","severity":"error","time":"2022-06-11T04:12:01Z"}
{"component":"entrypoint","error":"os: process already finished","file":"k8s.io/test-infra/prow/entrypoint/run.go:257","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Could not kill process after grace period","severity":"error","time":"2022-06-11T04:12:01Z"}