This job view page is being replaced by Spyglass soon. Check out the new job view.
PRhccheng72: [V2] fix: prevent requesting replica attachment in a row for one volume
ResultABORTED
Tests 0 failed / 0 succeeded
Started2023-01-17 17:54
Elapsed46m32s
Revisionac3d54e32c4e8daecb77c0bdd5dd97729c353957
Refs 1693

No Test Failures!


Error lines from build-log.txt

... skipping 244 lines ...

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 11345  100 11345    0     0   173k      0 --:--:-- --:--:-- --:--:--  173k
Downloading https://get.helm.sh/helm-v3.10.3-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
docker pull k8sprow.azurecr.io/azuredisk-csi:latest-v2-75eecb5c876f7e33167f6048a2d9266d5626d5cc || make container-all push-manifest
Error response from daemon: manifest for k8sprow.azurecr.io/azuredisk-csi:latest-v2-75eecb5c876f7e33167f6048a2d9266d5626d5cc not found: manifest unknown: manifest tagged by "latest-v2-75eecb5c876f7e33167f6048a2d9266d5626d5cc" is not found
make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=latest-v2-75eecb5c876f7e33167f6048a2d9266d5626d5cc -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=75eecb5c876f7e33167f6048a2d9266d5626d5cc -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2023-01-17T18:00:22Z -extldflags "-static"" -tags azurediskv2 -mod vendor -o _output/amd64/azurediskpluginv2.exe ./pkg/azurediskplugin
docker buildx rm container-builder || true
ERROR: no builder "container-builder" found
docker buildx create --use --name=container-builder
container-builder
# enable qemu for arm64 build
# https://github.com/docker/buildx/issues/464#issuecomment-741507760
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
Unable to find image 'tonistiigi/binfmt:latest' locally
... skipping 1514 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 60 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 254 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 108 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 359 lines ...
            - volumeName
            - volume_context
            - volume_id
            type: object
          status:
            description: status represents the current state of AzVolumeAttachment.
              includes error, state, and attachment status Required
            properties:
              detail:
                description: Status summarizes the current attachment state of the
                  volume attachment Nil Status indicates that the volume has not yet
                  been attached to the node
                properties:
... skipping 7 lines ...
                  role:
                    description: The current attachment role.
                    type: string
                required:
                - role
                type: object
              error:
                description: Error occurred during attach/detach of volume
                properties:
                  code:
                    type: string
                  message:
                    type: string
                  parameters:
... skipping 90 lines ...
            - volumeName
            - volume_context
            - volume_id
            type: object
          status:
            description: status represents the current state of AzVolumeAttachment.
              includes error, state, and attachment status
            properties:
              annotation:
                additionalProperties:
                  type: string
                description: Annotations contains additional resource information
                  to guide driver actions
... skipping 13 lines ...
                  role:
                    description: The current attachment role.
                    type: string
                required:
                - role
                type: object
              error:
                description: Error occurred during attach/detach of volume
                properties:
                  code:
                    type: string
                  message:
                    type: string
                  parameters:
... skipping 169 lines ...
            - maxMountReplicaCount
            - volumeCapability
            - volumeName
            type: object
          status:
            description: status represents the current state of AzVolume. includes
              error, state, and volume status
            properties:
              detail:
                description: Current status detail of the AzVolume Nil detail indicates
                  that the volume has not been created
                properties:
                  accessible_topology:
... skipping 28 lines ...
                    type: string
                required:
                - capacity_bytes
                - node_expansion_required
                - volume_id
                type: object
              error:
                description: Error occurred during creation/deletion of volume
                properties:
                  code:
                    type: string
                  message:
                    type: string
                  parameters:
... skipping 154 lines ...
            - maxMountReplicaCount
            - volumeCapability
            - volumeName
            type: object
          status:
            description: status represents the current state of AzVolume. includes
              error, state, and volume status
            properties:
              annotation:
                additionalProperties:
                  type: string
                description: Annotations contains additional resource information
                  to guide driver actions
... skipping 34 lines ...
                    type: string
                required:
                - capacity_bytes
                - node_expansion_required
                - volume_id
                type: object
              error:
                description: Error occurred during creation/deletion of volume
                properties:
                  code:
                    type: string
                  message:
                    type: string
                  parameters:
... skipping 1069 lines ...
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - "--leader-election-namespace=kube-system"
            - '-handle-volume-inuse-error=false'
            - '-feature-gates=RecoverVolumeExpansionFailure=true'
            - "-timeout=240s"
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
... skipping 345 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode
  test/e2e/storage/framework/testsuite.go:50
    should fail to use a volume in a pod with mismatched mode [Slow] [BeforeEach]
    test/e2e/storage/testsuites/volumemode.go:299

    Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping

    test/e2e/storage/external/external.go:262
------------------------------
... skipping 12 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
    test/e2e/storage/testsuites/subpath.go:258

    Distro debian doesn't support ntfs -- skipping

    test/e2e/storage/framework/testsuite.go:127
------------------------------
... skipping 74 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Inline-volume (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
    test/e2e/storage/testsuites/subpath.go:280

    Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping

    test/e2e/storage/external/external.go:262
------------------------------
... skipping 8 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Pre-provisioned PV (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
    test/e2e/storage/testsuites/subpath.go:269

    Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping

    test/e2e/storage/external/external.go:262
------------------------------
... skipping 287 lines ...
Jan 17 18:10:35.103: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comzkmgd] to have phase Bound
Jan 17 18:10:35.210: INFO: PersistentVolumeClaim test.csi.azure.comzkmgd found but phase is Pending instead of Bound.
Jan 17 18:10:37.319: INFO: PersistentVolumeClaim test.csi.azure.comzkmgd found but phase is Pending instead of Bound.
Jan 17 18:10:39.433: INFO: PersistentVolumeClaim test.csi.azure.comzkmgd found and phase=Bound (4.32946044s)
STEP: Creating pod pod-subpath-test-dynamicpv-r84q
STEP: Creating a pod to test subpath
Jan 17 18:10:39.763: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-r84q" in namespace "provisioning-5823" to be "Succeeded or Failed"
Jan 17 18:10:39.871: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 108.109087ms
Jan 17 18:10:41.984: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221264483s
Jan 17 18:10:44.095: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332119679s
Jan 17 18:10:46.207: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.444124411s
Jan 17 18:10:48.319: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.555838922s
Jan 17 18:10:50.427: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.664444081s
... skipping 24 lines ...
Jan 17 18:11:43.203: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.439809784s
Jan 17 18:11:45.312: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m5.548978429s
Jan 17 18:11:47.426: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m7.663213253s
Jan 17 18:11:49.536: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Pending", Reason="", readiness=false. Elapsed: 1m9.773363318s
Jan 17 18:11:51.644: INFO: Pod "pod-subpath-test-dynamicpv-r84q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m11.881016342s
STEP: Saw pod success
Jan 17 18:11:51.644: INFO: Pod "pod-subpath-test-dynamicpv-r84q" satisfied condition "Succeeded or Failed"
Jan 17 18:11:51.755: INFO: Trying to get logs from node k8s-agentpool1-19222836-vmss000002 pod pod-subpath-test-dynamicpv-r84q container test-container-volume-dynamicpv-r84q: <nil>
STEP: delete the pod
Jan 17 18:11:51.982: INFO: Waiting for pod pod-subpath-test-dynamicpv-r84q to disappear
Jan 17 18:11:52.094: INFO: Pod pod-subpath-test-dynamicpv-r84q no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-r84q
Jan 17 18:11:52.094: INFO: Deleting pod "pod-subpath-test-dynamicpv-r84q" in namespace "provisioning-5823"
... skipping 23 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should support existing directory
    test/e2e/storage/testsuites/subpath.go:207
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory","total":35,"completed":1,"skipped":272,"failed":0}

SSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes 
  should store data
  test/e2e/storage/testsuites/volumes.go:161
... skipping 103 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  test/e2e/storage/framework/testsuite.go:50
    should store data
    test/e2e/storage/testsuites/volumes.go:161
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","total":24,"completed":1,"skipped":72,"failed":0}

SS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] volumes
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:13:13.907: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 34 lines ...
[It] should check snapshot fields, check restore correctly works, check deletion (ephemeral)
  test/e2e/storage/testsuites/snapshottable.go:177
Jan 17 18:10:33.026: INFO: Creating resource for dynamic PV
Jan 17 18:10:33.026: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} 
STEP: creating a StorageClass snapshotting-2248-e2e-scdgl6w
STEP: [init] starting a pod to use the claim
Jan 17 18:10:33.262: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-9xvb2" in namespace "snapshotting-2248" to be "Succeeded or Failed"
Jan 17 18:10:33.407: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 144.413782ms
Jan 17 18:10:35.515: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.253143366s
Jan 17 18:10:37.636: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.373616082s
Jan 17 18:10:39.747: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.484334925s
Jan 17 18:10:41.859: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.596304833s
Jan 17 18:10:43.971: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.708529822s
Jan 17 18:10:46.086: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 12.823264412s
Jan 17 18:10:48.195: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 14.933125932s
Jan 17 18:10:50.305: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 17.042965966s
Jan 17 18:10:52.415: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 19.152956475s
Jan 17 18:10:54.528: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Pending", Reason="", readiness=false. Elapsed: 21.265448022s
Jan 17 18:10:56.642: INFO: Pod "pvc-snapshottable-tester-9xvb2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.37953349s
STEP: Saw pod success
Jan 17 18:10:56.642: INFO: Pod "pvc-snapshottable-tester-9xvb2" satisfied condition "Succeeded or Failed"
STEP: [init] checking the claim
STEP: creating a SnapshotClass
STEP: creating a dynamic VolumeSnapshot
Jan 17 18:10:57.102: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-5qgx4 to become ready
Jan 17 18:10:57.217: INFO: VolumeSnapshot snapshot-5qgx4 found but is not ready.
Jan 17 18:10:59.337: INFO: VolumeSnapshot snapshot-5qgx4 found but is not ready.
... skipping 40 lines ...
Jan 17 18:12:27.315: INFO: volumesnapshotcontents snapcontent-a091781d-efc8-47ba-bdaa-58d6a316b2f5 has been found and is not deleted
Jan 17 18:12:28.424: INFO: volumesnapshotcontents snapcontent-a091781d-efc8-47ba-bdaa-58d6a316b2f5 has been found and is not deleted
Jan 17 18:12:29.534: INFO: volumesnapshotcontents snapcontent-a091781d-efc8-47ba-bdaa-58d6a316b2f5 has been found and is not deleted
Jan 17 18:12:30.647: INFO: volumesnapshotcontents snapcontent-a091781d-efc8-47ba-bdaa-58d6a316b2f5 has been found and is not deleted
Jan 17 18:12:31.757: INFO: volumesnapshotcontents snapcontent-a091781d-efc8-47ba-bdaa-58d6a316b2f5 has been found and is not deleted
Jan 17 18:12:32.866: INFO: volumesnapshotcontents snapcontent-a091781d-efc8-47ba-bdaa-58d6a316b2f5 has been found and is not deleted
Jan 17 18:12:33.866: INFO: WaitUntil failed after reaching the timeout 30s
[AfterEach] volume snapshot controller
  test/e2e/storage/testsuites/snapshottable.go:172
Jan 17 18:12:34.005: INFO: Pod restored-pvc-tester-tx4sn has the following logs: 
Jan 17 18:12:34.005: INFO: Deleting pod "restored-pvc-tester-tx4sn" in namespace "snapshotting-2248"
Jan 17 18:12:34.114: INFO: Wait up to 5m0s for pod "restored-pvc-tester-tx4sn" to be fully deleted
Jan 17 18:13:06.339: INFO: deleting snapshot "snapshotting-2248"/"snapshot-5qgx4"
... skipping 26 lines ...
    test/e2e/storage/testsuites/snapshottable.go:113
      
      test/e2e/storage/testsuites/snapshottable.go:176
        should check snapshot fields, check restore correctly works, check deletion (ephemeral)
        test/e2e/storage/testsuites/snapshottable.go:177
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Ephemeral Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works, check deletion (ephemeral)","total":31,"completed":1,"skipped":70,"failed":0}

SSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  
  should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
  test/e2e/storage/testsuites/snapshottable.go:278
... skipping 20 lines ...
Jan 17 18:10:33.082: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comslk87] to have phase Bound
Jan 17 18:10:33.191: INFO: PersistentVolumeClaim test.csi.azure.comslk87 found but phase is Pending instead of Bound.
Jan 17 18:10:35.299: INFO: PersistentVolumeClaim test.csi.azure.comslk87 found but phase is Pending instead of Bound.
Jan 17 18:10:37.406: INFO: PersistentVolumeClaim test.csi.azure.comslk87 found and phase=Bound (4.324139871s)
STEP: [init] starting a pod to use the claim
STEP: [init] check pod success
Jan 17 18:10:37.846: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-7bcr2" in namespace "snapshotting-8937" to be "Succeeded or Failed"
Jan 17 18:10:37.958: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 111.969845ms
Jan 17 18:10:40.068: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221432275s
Jan 17 18:10:42.179: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332352068s
Jan 17 18:10:44.290: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.443874924s
Jan 17 18:10:46.403: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.556481234s
Jan 17 18:10:48.511: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.665006124s
Jan 17 18:10:50.619: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 12.772986097s
Jan 17 18:10:52.728: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 14.881756758s
Jan 17 18:10:54.838: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Pending", Reason="", readiness=false. Elapsed: 16.991907978s
Jan 17 18:10:56.951: INFO: Pod "pvc-snapshottable-tester-7bcr2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 19.104044026s
STEP: Saw pod success
Jan 17 18:10:56.951: INFO: Pod "pvc-snapshottable-tester-7bcr2" satisfied condition "Succeeded or Failed"
STEP: [init] checking the claim
Jan 17 18:10:57.063: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comslk87] to have phase Bound
Jan 17 18:10:57.174: INFO: PersistentVolumeClaim test.csi.azure.comslk87 found and phase=Bound (111.176235ms)
STEP: [init] checking the PV
STEP: [init] deleting the pod
Jan 17 18:10:57.534: INFO: Pod pvc-snapshottable-tester-7bcr2 has the following logs: 
... skipping 14 lines ...
Jan 17 18:11:07.162: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-88145030-740d-4901-b061-631e513f9a18 creationTime:2023-01-17T18:11:02Z readyToUse:true restoreSize:5Gi]
Jan 17 18:11:07.162: INFO: snapshotContentName snapcontent-88145030-740d-4901-b061-631e513f9a18
STEP: checking the snapshot
STEP: checking the SnapshotContent
STEP: Modifying source data test
STEP: modifying the data in the source PVC
Jan 17 18:11:07.596: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-mgkjx" in namespace "snapshotting-8937" to be "Succeeded or Failed"
Jan 17 18:11:07.703: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 106.791556ms
Jan 17 18:11:09.814: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218484308s
Jan 17 18:11:11.922: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 4.326373063s
Jan 17 18:11:14.030: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 6.43429064s
Jan 17 18:11:16.143: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 8.547025096s
Jan 17 18:11:18.252: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 10.655733798s
... skipping 33 lines ...
Jan 17 18:12:29.939: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.342714561s
Jan 17 18:12:32.046: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.450524195s
Jan 17 18:12:34.154: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.558113253s
Jan 17 18:12:36.267: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.671031443s
Jan 17 18:12:38.375: INFO: Pod "pvc-snapshottable-data-tester-mgkjx": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m30.779353084s
STEP: Saw pod success
Jan 17 18:12:38.375: INFO: Pod "pvc-snapshottable-data-tester-mgkjx" satisfied condition "Succeeded or Failed"
Jan 17 18:12:38.599: INFO: Pod pvc-snapshottable-data-tester-mgkjx has the following logs: 
Jan 17 18:12:38.599: INFO: Deleting pod "pvc-snapshottable-data-tester-mgkjx" in namespace "snapshotting-8937"
Jan 17 18:12:38.709: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-mgkjx" to be fully deleted
STEP: creating a pvc from the snapshot
STEP: starting a pod to use the snapshot
Jan 17 18:13:29.252: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-jsprhnfx.westeurope.cloudapp.azure.com --kubeconfig=/root/tmp2722049285/kubeconfig/kubeconfig.westeurope.json --namespace=snapshotting-8937 exec restored-pvc-tester-wgcjt --namespace=snapshotting-8937 -- cat /mnt/test/data'
... skipping 47 lines ...
    test/e2e/storage/testsuites/snapshottable.go:113
      
      test/e2e/storage/testsuites/snapshottable.go:176
        should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
        test/e2e/storage/testsuites/snapshottable.go:278
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)","total":46,"completed":1,"skipped":40,"failed":0}

SSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:14:15.348: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 74 lines ...
  test/e2e/framework/framework.go:187
STEP: Creating a kubernetes client
Jan 17 18:14:16.465: INFO: >>> kubeConfig: /root/tmp2722049285/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename topology
STEP: Waiting for a default service account to be provisioned in namespace
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  test/e2e/storage/testsuites/topology.go:194
Jan 17 18:14:17.211: INFO: Driver didn't provide topology keys -- skipping
[AfterEach] [Testpattern: Dynamic PV (delayed binding)] topology
  test/e2e/framework/framework.go:188
Jan 17 18:14:17.212: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "topology-8961" for this suite.


S [SKIPPING] [0.966 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (delayed binding)] topology
  test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement]
    test/e2e/storage/testsuites/topology.go:194

    Driver didn't provide topology keys -- skipping

    test/e2e/storage/testsuites/topology.go:126
------------------------------
... skipping 342 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] provisioning
  test/e2e/storage/framework/testsuite.go:50
    should provision storage with pvc data source in parallel [Slow]
    test/e2e/storage/testsuites/provisioning.go:459
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with pvc data source in parallel [Slow]","total":34,"completed":1,"skipped":218,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:14:19.571: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 142 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (ext4)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should concurrently access the single read-only volume from pods on the same node
    test/e2e/storage/testsuites/multivolume.go:423
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node","total":31,"completed":1,"skipped":72,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:14:28.018: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 220 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should access to two volumes with the same volume mode and retain data across pod recreation on different node
    test/e2e/storage/testsuites/multivolume.go:168
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node","total":29,"completed":1,"skipped":30,"failed":0}

SSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:14:33.483: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 307 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    test/e2e/storage/testsuites/volumemode.go:354
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":34,"completed":2,"skipped":292,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:16:48.628: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 149 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]
    test/e2e/storage/testsuites/multivolume.go:378
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":35,"completed":2,"skipped":282,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning 
  should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
  test/e2e/storage/testsuites/provisioning.go:208
... skipping 138 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] provisioning
  test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
    test/e2e/storage/testsuites/provisioning.go:208
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":24,"completed":2,"skipped":94,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] 
  should access to two volumes with different volume mode and retain data across pod recreation on different node
  test/e2e/storage/testsuites/multivolume.go:248
... skipping 194 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should access to two volumes with different volume mode and retain data across pod recreation on different node
    test/e2e/storage/testsuites/multivolume.go:248
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node","total":46,"completed":2,"skipped":154,"failed":0}

SSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
  test/e2e/storage/testsuites/fsgroupchangepolicy.go:216
... skipping 91 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    test/e2e/storage/testsuites/fsgroupchangepolicy.go:216
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":29,"completed":2,"skipped":113,"failed":0}

SSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support existing directories when readOnly specified in the volumeSource
  test/e2e/storage/testsuites/subpath.go:397

{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the single volume from pods on the same node","total":31,"completed":2,"skipped":73,"failed":0}
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/framework/framework.go:187
STEP: Creating a kubernetes client
Jan 17 18:15:23.021: INFO: >>> kubeConfig: /root/tmp2722049285/kubeconfig/kubeconfig.westeurope.json
... skipping 10 lines ...
Jan 17 18:15:23.999: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comxknwt] to have phase Bound
Jan 17 18:15:24.107: INFO: PersistentVolumeClaim test.csi.azure.comxknwt found but phase is Pending instead of Bound.
Jan 17 18:15:26.217: INFO: PersistentVolumeClaim test.csi.azure.comxknwt found but phase is Pending instead of Bound.
Jan 17 18:15:28.326: INFO: PersistentVolumeClaim test.csi.azure.comxknwt found and phase=Bound (4.32730575s)
STEP: Creating pod pod-subpath-test-dynamicpv-s5h5
STEP: Creating a pod to test subpath
Jan 17 18:15:28.653: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-s5h5" in namespace "provisioning-2018" to be "Succeeded or Failed"
Jan 17 18:15:28.762: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 108.405401ms
Jan 17 18:15:30.871: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.217089276s
Jan 17 18:15:32.980: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.326963426s
Jan 17 18:15:35.093: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.439398857s
Jan 17 18:15:37.202: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.54886802s
Jan 17 18:15:39.311: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.657538745s
... skipping 8 lines ...
Jan 17 18:15:58.292: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 29.638323561s
Jan 17 18:16:00.401: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 31.747225627s
Jan 17 18:16:02.511: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 33.857540379s
Jan 17 18:16:04.620: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 35.966865732s
Jan 17 18:16:06.739: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.0857365s
STEP: Saw pod success
Jan 17 18:16:06.739: INFO: Pod "pod-subpath-test-dynamicpv-s5h5" satisfied condition "Succeeded or Failed"
Jan 17 18:16:06.848: INFO: Trying to get logs from node k8s-agentpool1-19222836-vmss000000 pod pod-subpath-test-dynamicpv-s5h5 container test-container-subpath-dynamicpv-s5h5: <nil>
STEP: delete the pod
Jan 17 18:16:07.105: INFO: Waiting for pod pod-subpath-test-dynamicpv-s5h5 to disappear
Jan 17 18:16:07.213: INFO: Pod pod-subpath-test-dynamicpv-s5h5 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-s5h5
Jan 17 18:16:07.213: INFO: Deleting pod "pod-subpath-test-dynamicpv-s5h5" in namespace "provisioning-2018"
STEP: Creating pod pod-subpath-test-dynamicpv-s5h5
STEP: Creating a pod to test subpath
Jan 17 18:16:07.434: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-s5h5" in namespace "provisioning-2018" to be "Succeeded or Failed"
Jan 17 18:16:07.543: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 108.211143ms
Jan 17 18:16:09.652: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.217441185s
Jan 17 18:16:11.761: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.326653936s
Jan 17 18:16:13.871: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.436400404s
Jan 17 18:16:15.993: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.558861452s
Jan 17 18:16:18.103: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.66902428s
... skipping 16 lines ...
Jan 17 18:16:53.970: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 46.535938521s
Jan 17 18:16:56.090: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 48.655762425s
Jan 17 18:16:58.199: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 50.76500279s
Jan 17 18:17:00.364: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Pending", Reason="", readiness=false. Elapsed: 52.929992683s
Jan 17 18:17:02.475: INFO: Pod "pod-subpath-test-dynamicpv-s5h5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 55.04062991s
STEP: Saw pod success
Jan 17 18:17:02.475: INFO: Pod "pod-subpath-test-dynamicpv-s5h5" satisfied condition "Succeeded or Failed"
Jan 17 18:17:02.587: INFO: Trying to get logs from node k8s-agentpool1-19222836-vmss000000 pod pod-subpath-test-dynamicpv-s5h5 container test-container-subpath-dynamicpv-s5h5: <nil>
STEP: delete the pod
Jan 17 18:17:02.816: INFO: Waiting for pod pod-subpath-test-dynamicpv-s5h5 to disappear
Jan 17 18:17:02.925: INFO: Pod pod-subpath-test-dynamicpv-s5h5 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-s5h5
Jan 17 18:17:02.925: INFO: Deleting pod "pod-subpath-test-dynamicpv-s5h5" in namespace "provisioning-2018"
... skipping 29 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should support existing directories when readOnly specified in the volumeSource
    test/e2e/storage/testsuites/subpath.go:397
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","total":31,"completed":3,"skipped":73,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits 
  should verify that all csinodes have volume limits
  test/e2e/storage/testsuites/volumelimits.go:249
... skipping 16 lines ...
  test/e2e/framework/framework.go:188
Jan 17 18:18:16.730: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volumelimits-4055" for this suite.

•
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits","total":31,"completed":4,"skipped":96,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode 
  should not mount / map unused volumes in a pod [LinuxOnly]
  test/e2e/storage/testsuites/volumemode.go:354
... skipping 106 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumeMode
  test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    test/e2e/storage/testsuites/volumemode.go:354
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":31,"completed":2,"skipped":103,"failed":0}

SSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:19:04.744: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 258 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should access to two volumes with different volume mode and retain data across pod recreation on the same node
    test/e2e/storage/testsuites/multivolume.go:209
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node","total":34,"completed":3,"skipped":370,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:19:11.862: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 3 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.002 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Inline-volume (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
    test/e2e/storage/testsuites/subpath.go:269

    Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping

    test/e2e/storage/external/external.go:262
------------------------------
... skipping 29 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Inline-volume (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
    test/e2e/storage/testsuites/subpath.go:258

    Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping

    test/e2e/storage/external/external.go:262
------------------------------
... skipping 150 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents
    test/e2e/storage/testsuites/fsgroupchangepolicy.go:216
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents","total":29,"completed":3,"skipped":134,"failed":0}

S
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:20:03.356: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 104 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  test/e2e/storage/framework/testsuite.go:50
    should support two pods which have the same volume definition
    test/e2e/storage/testsuites/ephemeral.go:216
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which have the same volume definition","total":35,"completed":3,"skipped":306,"failed":0}

SS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:20:23.897: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 22 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Inline-volume (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach]
    test/e2e/storage/testsuites/subpath.go:242

    Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping

    test/e2e/storage/external/external.go:262
------------------------------
S
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should fail if subpath file is outside the volume [Slow][LinuxOnly]
  test/e2e/storage/testsuites/subpath.go:258

[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/framework/framework.go:187
STEP: Creating a kubernetes client
Jan 17 18:17:46.351: INFO: >>> kubeConfig: /root/tmp2722049285/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename provisioning
STEP: Waiting for a default service account to be provisioned in namespace
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace
[It] should fail if subpath file is outside the volume [Slow][LinuxOnly]
  test/e2e/storage/testsuites/subpath.go:258
Jan 17 18:17:47.102: INFO: Creating resource for dynamic PV
Jan 17 18:17:47.102: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-5472-e2e-sc5wjnz
STEP: creating a claim
Jan 17 18:17:47.218: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Jan 17 18:17:47.327: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comdr4l8] to have phase Bound
Jan 17 18:17:47.434: INFO: PersistentVolumeClaim test.csi.azure.comdr4l8 found but phase is Pending instead of Bound.
Jan 17 18:17:49.542: INFO: PersistentVolumeClaim test.csi.azure.comdr4l8 found but phase is Pending instead of Bound.
Jan 17 18:17:51.659: INFO: PersistentVolumeClaim test.csi.azure.comdr4l8 found and phase=Bound (4.331007573s)
STEP: Creating pod pod-subpath-test-dynamicpv-kgmw
STEP: Checking for subpath error in container status
Jan 17 18:19:14.213: INFO: Deleting pod "pod-subpath-test-dynamicpv-kgmw" in namespace "provisioning-5472"
Jan 17 18:19:14.324: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-kgmw" to be fully deleted
STEP: Deleting pod
Jan 17 18:19:16.543: INFO: Deleting pod "pod-subpath-test-dynamicpv-kgmw" in namespace "provisioning-5472"
STEP: Deleting pvc
Jan 17 18:19:16.651: INFO: Deleting PersistentVolumeClaim "test.csi.azure.comdr4l8"
... skipping 22 lines ...

• [SLOW TEST:162.598 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if subpath file is outside the volume [Slow][LinuxOnly]
    test/e2e/storage/testsuites/subpath.go:258
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath file is outside the volume [Slow][LinuxOnly]","total":46,"completed":3,"skipped":163,"failed":0}

SSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:20:28.975: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 94 lines ...
[It] should check snapshot fields, check restore correctly works, check deletion (ephemeral)
  test/e2e/storage/testsuites/snapshottable.go:177
Jan 17 18:19:05.679: INFO: Creating resource for dynamic PV
Jan 17 18:19:05.679: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} 
STEP: creating a StorageClass snapshotting-41-e2e-sccssmx
STEP: [init] starting a pod to use the claim
Jan 17 18:19:05.900: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-rs2db" in namespace "snapshotting-41" to be "Succeeded or Failed"
Jan 17 18:19:06.009: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 109.067721ms
Jan 17 18:19:08.118: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218261642s
Jan 17 18:19:10.228: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 4.328279677s
Jan 17 18:19:12.337: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 6.437217373s
Jan 17 18:19:14.447: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 8.547208771s
Jan 17 18:19:16.557: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 10.656953545s
Jan 17 18:19:18.667: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 12.767284149s
Jan 17 18:19:20.776: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 14.875944983s
Jan 17 18:19:22.885: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Pending", Reason="", readiness=false. Elapsed: 16.985311771s
Jan 17 18:19:24.995: INFO: Pod "pvc-snapshottable-tester-rs2db": Phase="Succeeded", Reason="", readiness=false. Elapsed: 19.094988107s
STEP: Saw pod success
Jan 17 18:19:24.995: INFO: Pod "pvc-snapshottable-tester-rs2db" satisfied condition "Succeeded or Failed"
STEP: [init] checking the claim
STEP: creating a SnapshotClass
STEP: creating a dynamic VolumeSnapshot
Jan 17 18:19:25.434: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-ss9zr to become ready
Jan 17 18:19:25.543: INFO: VolumeSnapshot snapshot-ss9zr found but is not ready.
Jan 17 18:19:27.652: INFO: VolumeSnapshot snapshot-ss9zr found but is not ready.
... skipping 49 lines ...
    test/e2e/storage/testsuites/snapshottable.go:113
      
      test/e2e/storage/testsuites/snapshottable.go:176
        should check snapshot fields, check restore correctly works, check deletion (ephemeral)
        test/e2e/storage/testsuites/snapshottable.go:177
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Ephemeral Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works, check deletion (ephemeral)","total":31,"completed":3,"skipped":221,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] volumeMode
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:21:05.273: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 94 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    test/e2e/storage/testsuites/ephemeral.go:196
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume","total":31,"completed":5,"skipped":122,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] volumes 
  should store data
  test/e2e/storage/testsuites/volumes.go:161
... skipping 108 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (xfs)][Slow] volumes
  test/e2e/storage/framework/testsuite.go:50
    should store data
    test/e2e/storage/testsuites/volumes.go:161
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] volumes should store data","total":35,"completed":4,"skipped":311,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:22:02.900: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 102 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should concurrently access the single read-only volume from pods on the same node
    test/e2e/storage/testsuites/multivolume.go:423
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node","total":34,"completed":4,"skipped":579,"failed":0}

SSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  
  should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
  test/e2e/storage/testsuites/snapshottable.go:278
... skipping 17 lines ...
Jan 17 18:17:43.019: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comrblwt] to have phase Bound
Jan 17 18:17:43.126: INFO: PersistentVolumeClaim test.csi.azure.comrblwt found but phase is Pending instead of Bound.
Jan 17 18:17:45.251: INFO: PersistentVolumeClaim test.csi.azure.comrblwt found but phase is Pending instead of Bound.
Jan 17 18:17:47.359: INFO: PersistentVolumeClaim test.csi.azure.comrblwt found and phase=Bound (4.34067245s)
STEP: [init] starting a pod to use the claim
STEP: [init] check pod success
Jan 17 18:17:47.797: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-td948" in namespace "snapshotting-4384" to be "Succeeded or Failed"
Jan 17 18:17:47.905: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 107.433074ms
Jan 17 18:17:50.017: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 2.219662294s
Jan 17 18:17:52.125: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 4.327994361s
Jan 17 18:17:54.233: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 6.435997609s
Jan 17 18:17:56.342: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 8.544791337s
Jan 17 18:17:58.451: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 10.653218626s
Jan 17 18:18:00.560: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 12.762834963s
Jan 17 18:18:02.669: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 14.871886151s
Jan 17 18:18:04.779: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 16.981543288s
Jan 17 18:18:06.890: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Pending", Reason="", readiness=false. Elapsed: 19.092401672s
Jan 17 18:18:08.999: INFO: Pod "pvc-snapshottable-tester-td948": Phase="Succeeded", Reason="", readiness=false. Elapsed: 21.201904492s
STEP: Saw pod success
Jan 17 18:18:08.999: INFO: Pod "pvc-snapshottable-tester-td948" satisfied condition "Succeeded or Failed"
STEP: [init] checking the claim
Jan 17 18:18:09.107: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comrblwt] to have phase Bound
Jan 17 18:18:09.214: INFO: PersistentVolumeClaim test.csi.azure.comrblwt found and phase=Bound (107.228266ms)
STEP: [init] checking the PV
STEP: [init] deleting the pod
Jan 17 18:18:09.562: INFO: Pod pvc-snapshottable-tester-td948 has the following logs: 
... skipping 33 lines ...
Jan 17 18:18:17.965: INFO: WaitUntil finished successfully after 108.070265ms
STEP: getting the snapshot and snapshot content
STEP: checking the snapshot
STEP: checking the SnapshotContent
STEP: Modifying source data test
STEP: modifying the data in the source PVC
Jan 17 18:18:18.505: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-pl779" in namespace "snapshotting-4384" to be "Succeeded or Failed"
Jan 17 18:18:18.614: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 109.105414ms
Jan 17 18:18:20.723: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 2.217953877s
Jan 17 18:18:22.832: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 4.327132436s
Jan 17 18:18:24.941: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 6.436117858s
Jan 17 18:18:27.049: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 8.544552108s
Jan 17 18:18:29.159: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 10.653808847s
... skipping 90 lines ...
Jan 17 18:21:41.102: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.59689001s
Jan 17 18:21:43.210: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.704864975s
Jan 17 18:21:45.318: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.81352431s
Jan 17 18:21:47.427: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.921979362s
Jan 17 18:21:49.535: INFO: Pod "pvc-snapshottable-data-tester-pl779": Phase="Succeeded", Reason="", readiness=false. Elapsed: 3m31.030185286s
STEP: Saw pod success
Jan 17 18:21:49.535: INFO: Pod "pvc-snapshottable-data-tester-pl779" satisfied condition "Succeeded or Failed"
Jan 17 18:21:49.772: INFO: Pod pvc-snapshottable-data-tester-pl779 has the following logs: 
Jan 17 18:21:49.772: INFO: Deleting pod "pvc-snapshottable-data-tester-pl779" in namespace "snapshotting-4384"
Jan 17 18:21:49.890: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-pl779" to be fully deleted
STEP: creating a pvc from the snapshot
STEP: starting a pod to use the snapshot
Jan 17 18:22:20.431: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-jsprhnfx.westeurope.cloudapp.azure.com --kubeconfig=/root/tmp2722049285/kubeconfig/kubeconfig.westeurope.json --namespace=snapshotting-4384 exec restored-pvc-tester-9n47k --namespace=snapshotting-4384 -- cat /mnt/test/data'
... skipping 33 lines ...
Jan 17 18:22:46.686: INFO: volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e has been found and is not deleted
Jan 17 18:22:47.795: INFO: volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e has been found and is not deleted
Jan 17 18:22:48.903: INFO: volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e has been found and is not deleted
Jan 17 18:22:50.012: INFO: volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e has been found and is not deleted
Jan 17 18:22:51.120: INFO: volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e has been found and is not deleted
Jan 17 18:22:52.228: INFO: volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e has been found and is not deleted
Jan 17 18:22:53.228: INFO: WaitUntil failed after reaching the timeout 30s
[AfterEach] volume snapshot controller
  test/e2e/storage/testsuites/snapshottable.go:172
Jan 17 18:22:53.336: INFO: Error getting logs for pod restored-pvc-tester-9n47k: the server could not find the requested resource (get pods restored-pvc-tester-9n47k)
Jan 17 18:22:53.336: INFO: Deleting pod "restored-pvc-tester-9n47k" in namespace "snapshotting-4384"
Jan 17 18:22:53.443: INFO: deleting claim "snapshotting-4384"/"pvc-jfc49"
Jan 17 18:22:53.549: INFO: deleting snapshot "snapshotting-4384"/"pre-provisioned-snapshot-2e340b75-6018-4e04-b304-24f64537ec2e"
Jan 17 18:22:53.656: INFO: deleting snapshot content "pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e"
Jan 17 18:22:53.984: INFO: Waiting up to 5m0s for volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e to be deleted
Jan 17 18:22:54.091: INFO: volumesnapshotcontents pre-provisioned-snapcontent-2e340b75-6018-4e04-b304-24f64537ec2e has been found and is not deleted
... skipping 27 lines ...
    test/e2e/storage/testsuites/snapshottable.go:113
      
      test/e2e/storage/testsuites/snapshottable.go:176
        should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)
        test/e2e/storage/testsuites/snapshottable.go:278
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)","total":24,"completed":3,"skipped":149,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:23:11.638: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 86 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  test/e2e/storage/framework/testsuite.go:50
    should support restarting containers using directory as subpath [Slow]
    test/e2e/storage/testsuites/subpath.go:322
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support restarting containers using directory as subpath [Slow]","total":29,"completed":4,"skipped":270,"failed":0}

SSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
  test/e2e/storage/testsuites/fsgroupchangepolicy.go:216
... skipping 97 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  test/e2e/storage/framework/testsuite.go:50
    (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents
    test/e2e/storage/testsuites/fsgroupchangepolicy.go:216
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":31,"completed":6,"skipped":159,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should support multiple inline ephemeral volumes
  test/e2e/storage/testsuites/ephemeral.go:254
... skipping 51 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  test/e2e/storage/framework/testsuite.go:50
    should support multiple inline ephemeral volumes
    test/e2e/storage/testsuites/ephemeral.go:254
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":31,"completed":4,"skipped":363,"failed":0}

SSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:23:42.545: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 211 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (ext4)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should access to two volumes with the same volume mode and retain data across pod recreation on the same node
    test/e2e/storage/testsuites/multivolume.go:138
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node","total":35,"completed":5,"skipped":354,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning 
  should provision storage with pvc data source
  test/e2e/storage/testsuites/provisioning.go:421
... skipping 101 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] provisioning
  test/e2e/storage/framework/testsuite.go:50
    should provision storage with pvc data source
    test/e2e/storage/testsuites/provisioning.go:421
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with pvc data source","total":46,"completed":4,"skipped":216,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:24:28.684: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 24 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Pre-provisioned PV (block volmode)] volumeMode
  test/e2e/storage/framework/testsuite.go:50
    should fail to use a volume in a pod with mismatched mode [Slow] [BeforeEach]
    test/e2e/storage/testsuites/volumemode.go:299

    Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping

    test/e2e/storage/external/external.go:262
------------------------------
SSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode 
  should fail to use a volume in a pod with mismatched mode [Slow]
  test/e2e/storage/testsuites/volumemode.go:299

[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode
  test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode
  test/e2e/framework/framework.go:187
STEP: Creating a kubernetes client
Jan 17 18:23:11.641: INFO: >>> kubeConfig: /root/tmp2722049285/kubeconfig/kubeconfig.westeurope.json
STEP: Building a namespace api object, basename volumemode
STEP: Waiting for a default service account to be provisioned in namespace
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace
[It] should fail to use a volume in a pod with mismatched mode [Slow]
  test/e2e/storage/testsuites/volumemode.go:299
Jan 17 18:23:12.391: INFO: Creating resource for dynamic PV
Jan 17 18:23:12.391: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} 
STEP: creating a StorageClass volumemode-7038-e2e-scczq5q
STEP: creating a claim
Jan 17 18:23:12.608: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.combnqrf] to have phase Bound
Jan 17 18:23:12.715: INFO: PersistentVolumeClaim test.csi.azure.combnqrf found but phase is Pending instead of Bound.
Jan 17 18:23:14.823: INFO: PersistentVolumeClaim test.csi.azure.combnqrf found but phase is Pending instead of Bound.
Jan 17 18:23:16.931: INFO: PersistentVolumeClaim test.csi.azure.combnqrf found and phase=Bound (4.322543639s)
STEP: Creating pod
STEP: Waiting for the pod to fail
Jan 17 18:23:19.592: INFO: Deleting pod "pod-830ad734-7068-4a7d-9c78-51ea57161824" in namespace "volumemode-7038"
Jan 17 18:23:19.701: INFO: Wait up to 5m0s for pod "pod-830ad734-7068-4a7d-9c78-51ea57161824" to be fully deleted
STEP: Deleting pvc
Jan 17 18:23:21.919: INFO: Deleting PersistentVolumeClaim "test.csi.azure.combnqrf"
Jan 17 18:23:22.027: INFO: Waiting up to 5m0s for PersistentVolume pvc-ebbe7031-faf7-4dfc-82e8-693f60cfd0d3 to get deleted
Jan 17 18:23:22.134: INFO: PersistentVolume pvc-ebbe7031-faf7-4dfc-82e8-693f60cfd0d3 found and phase=Released (107.189918ms)
... skipping 20 lines ...

• [SLOW TEST:82.568 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (block volmode)] volumeMode
  test/e2e/storage/framework/testsuite.go:50
    should fail to use a volume in a pod with mismatched mode [Slow]
    test/e2e/storage/testsuites/volumemode.go:299
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode should fail to use a volume in a pod with mismatched mode [Slow]","total":24,"completed":4,"skipped":177,"failed":0}

SSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
  test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
... skipping 222 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:50
    should access to two volumes with the same volume mode and retain data across pod recreation on the same node
    test/e2e/storage/testsuites/multivolume.go:138
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node","total":34,"completed":5,"skipped":599,"failed":0}

SSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow]
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:24:57.566: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 155 lines ...
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy
  test/e2e/storage/framework/testsuite.go:50
    (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents
    test/e2e/storage/testsuites/fsgroupchangepolicy.go:216
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents","total":31,"completed":5,"skipped":389,"failed":0}

SSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  test/e2e/storage/framework/testsuite.go:51
Jan 17 18:25:27.976: INFO: Distro debian doesn't support ntfs -- skipping
... skipping 3 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  test/e2e/storage/framework/testsuite.go:50
    should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
    test/e2e/storage/testsuites/subpath.go:280

    Distro debian doesn't support ntfs -- skipping

    test/e2e/storage/framework/testsuite.go:127
------------------------------
... skipping 53 lines ...
        test/e2e/storage/testsuites/snapshottable.go:177

        volume type "DynamicPV" is not ephemeral

        test/e2e/storage/testsuites/snapshottable.go:179
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:168","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Entrypoint received interrupt: terminated","severity":"error","time":"2023-01-17T18:25:53Z"}
++ early_exit_handler
++ '[' -n 173 ']'
++ kill -TERM 173
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 4 lines ...