pull-csi-driver-smb-e2e-windows #1521393877724958720 Results

Test Failures

SMB CSI Driver End-to-End Tests Dynamic Provisioning [createSubDir] should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [smb.csi.k8s.io] [Windows] 7m6s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=SMB\sCSI\sDriver\sEnd\-to\-End\sTests\sDynamic\sProvisioning\s\[createSubDir\]\sshould\screate\smultiple\sPV\sobjects\,\sbind\sto\sPVCs\sand\sattach\sall\sto\sdifferent\spods\son\sthe\ssame\snode\s\[smb\.csi\.k8s\.io\]\s\[Windows\]$'

/home/prow/go/src/sigs.k8s.io/csi-driver-smb/test/e2e/dynamic_provisioning_test.go:170
Unexpected error:
    <*errors.errorString | 0xc0002eabc0>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred
/home/prow/go/src/sigs.k8s.io/csi-driver-smb/test/e2e/testsuites/testsuites.go:532
				�[1mSTEP�[0m: Creating a kubernetes client
May  3 08:31:25.090: INFO: >>> kubeConfig: /root/tmp3374271991/kubeconfig/kubeconfig.eastus.json
�[1mSTEP�[0m: Building a namespace api object, basename smb
�[1mSTEP�[0m: Binding the e2e-test-privileged-psp PodSecurityPolicy to the default service account in smb-3274
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
�[1mSTEP�[0m: setting up the StorageClass
�[1mSTEP�[0m: creating a StorageClass 
�[1mSTEP�[0m: setting up the PVC and PV
�[1mSTEP�[0m: creating a PVC
�[1mSTEP�[0m: waiting for PVC to be in phase "Bound"
May  3 08:31:25.825: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-v6kvj] to have phase Bound
May  3 08:31:25.857: INFO: PersistentVolumeClaim pvc-v6kvj found but phase is Pending instead of Bound.
May  3 08:31:27.891: INFO: PersistentVolumeClaim pvc-v6kvj found and phase=Bound (2.066018482s)
�[1mSTEP�[0m: checking the PVC
�[1mSTEP�[0m: validating provisioned PV
�[1mSTEP�[0m: checking the PV
�[1mSTEP�[0m: deploying the pod
�[1mSTEP�[0m: checking that the pod is running
�[1mSTEP�[0m: setting up the StorageClass
�[1mSTEP�[0m: creating a StorageClass 
�[1mSTEP�[0m: setting up the PVC and PV
�[1mSTEP�[0m: creating a PVC
�[1mSTEP�[0m: waiting for PVC to be in phase "Bound"
May  3 08:31:46.135: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-nxj5m] to have phase Bound
May  3 08:31:46.179: INFO: PersistentVolumeClaim pvc-nxj5m found but phase is Pending instead of Bound.
May  3 08:31:48.212: INFO: PersistentVolumeClaim pvc-nxj5m found and phase=Bound (2.076966542s)
�[1mSTEP�[0m: checking the PVC
�[1mSTEP�[0m: validating provisioned PV
�[1mSTEP�[0m: checking the PV
�[1mSTEP�[0m: deploying the pod
�[1mSTEP�[0m: checking that the pod is running
May  3 08:36:48.450: INFO: deleting Pod "smb-3274"/"smb-volume-tester-hp5hf"
May  3 08:36:48.674: INFO: Error getting logs for pod smb-volume-tester-hp5hf: the server rejected our request for an unknown reason (get pods smb-volume-tester-hp5hf)
�[1mSTEP�[0m: Deleting pod smb-volume-tester-hp5hf in namespace smb-3274
May  3 08:36:48.710: INFO: deleting PVC "smb-3274"/"pvc-nxj5m"
May  3 08:36:48.711: INFO: Deleting PersistentVolumeClaim "pvc-nxj5m"
�[1mSTEP�[0m: waiting for claim's PV "pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a" to be deleted
May  3 08:36:48.815: INFO: Waiting up to 10m0s for PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a to get deleted
May  3 08:36:48.847: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (32.086333ms)
May  3 08:36:53.880: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (5.065281317s)
May  3 08:36:58.913: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (10.098305679s)
May  3 08:37:03.948: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (15.132634175s)
May  3 08:37:08.981: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (20.165643843s)
May  3 08:37:14.014: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (25.19896514s)
May  3 08:37:19.051: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (30.235511083s)
May  3 08:37:24.084: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (35.268692925s)
May  3 08:37:29.118: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (40.302369551s)
May  3 08:37:34.154: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (45.338396732s)
May  3 08:37:39.190: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (50.375027966s)
May  3 08:37:44.228: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (55.412390655s)
May  3 08:37:49.260: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (1m0.445223929s)
May  3 08:37:54.298: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (1m5.483274223s)
May  3 08:37:59.339: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (1m10.523524911s)
May  3 08:38:04.373: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (1m15.557363924s)
May  3 08:38:09.407: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (1m20.592048736s)
May  3 08:38:14.444: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (1m25.628787422s)
May  3 08:38:19.481: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a found and phase=Bound (1m30.666315451s)
May  3 08:38:24.518: INFO: PersistentVolume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a was removed
May  3 08:38:24.518: INFO: Waiting up to 5m0s for PersistentVolumeClaim smb-3274 to be removed
May  3 08:38:24.556: INFO: Claim "smb-3274" in namespace "pvc-nxj5m" doesn't exist in the system
May  3 08:38:24.556: INFO: deleting StorageClass smb-3274-smb.csi.k8s.io-dynamic-sc-qmgc7
May  3 08:38:24.596: INFO: deleting Pod "smb-3274"/"smb-volume-tester-lmc75"
May  3 08:38:24.684: INFO: Pod smb-volume-tester-lmc75 has the following logs: 
�[1mSTEP�[0m: Deleting pod smb-volume-tester-lmc75 in namespace smb-3274
May  3 08:38:24.723: INFO: deleting PVC "smb-3274"/"pvc-v6kvj"
May  3 08:38:24.723: INFO: Deleting PersistentVolumeClaim "pvc-v6kvj"
�[1mSTEP�[0m: waiting for claim's PV "pvc-3d77ab9e-9216-4734-9f2c-07bd143217a5" to be deleted
May  3 08:38:24.850: INFO: Waiting up to 10m0s for PersistentVolume pvc-3d77ab9e-9216-4734-9f2c-07bd143217a5 to get deleted
May  3 08:38:24.882: INFO: PersistentVolume pvc-3d77ab9e-9216-4734-9f2c-07bd143217a5 found and phase=Bound (31.752084ms)
May  3 08:38:29.918: INFO: PersistentVolume pvc-3d77ab9e-9216-4734-9f2c-07bd143217a5 was removed
May  3 08:38:29.918: INFO: Waiting up to 5m0s for PersistentVolumeClaim smb-3274 to be removed
May  3 08:38:29.950: INFO: Claim "smb-3274" in namespace "pvc-v6kvj" doesn't exist in the system
May  3 08:38:29.950: INFO: deleting StorageClass smb-3274-smb.csi.k8s.io-dynamic-sc-47bmc
�[1mSTEP�[0m: Collecting events from namespace "smb-3274".
�[1mSTEP�[0m: Found 15 events.
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:25 +0000 UTC - event for pvc-v6kvj: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } Provisioning: External provisioner is provisioning volume for claim "smb-3274/pvc-v6kvj"
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:25 +0000 UTC - event for pvc-v6kvj: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "smb.csi.k8s.io" or manually created by system administrator
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:25 +0000 UTC - event for pvc-v6kvj: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } ProvisioningSucceeded: Successfully provisioned volume pvc-3d77ab9e-9216-4734-9f2c-07bd143217a5
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:27 +0000 UTC - event for smb-volume-tester-lmc75: {default-scheduler } Scheduled: Successfully assigned smb-3274/smb-volume-tester-lmc75 to 2935k8s001
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:33 +0000 UTC - event for smb-volume-tester-lmc75: {kubelet 2935k8s001} Pulling: Pulling image "e2eteam/busybox:1.29"
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:43 +0000 UTC - event for smb-volume-tester-lmc75: {kubelet 2935k8s001} Pulled: Successfully pulled image "e2eteam/busybox:1.29" in 9.5580368s
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:43 +0000 UTC - event for smb-volume-tester-lmc75: {kubelet 2935k8s001} Created: Created container volume-tester
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:44 +0000 UTC - event for smb-volume-tester-lmc75: {kubelet 2935k8s001} Started: Started container volume-tester
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:46 +0000 UTC - event for pvc-nxj5m: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } Provisioning: External provisioner is provisioning volume for claim "smb-3274/pvc-nxj5m"
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:46 +0000 UTC - event for pvc-nxj5m: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "smb.csi.k8s.io" or manually created by system administrator
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:46 +0000 UTC - event for pvc-nxj5m: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } ProvisioningSucceeded: Successfully provisioned volume pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:48 +0000 UTC - event for smb-volume-tester-hp5hf: {kubelet 2935k8s000} FailedMount: MountVolume.MountDevice failed for volume "pvc-ca0e2493-fbd7-4840-b20b-4c392358ce1a" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name smb.csi.k8s.io not found in the list of registered CSI drivers
May  3 08:38:30.024: INFO: At 2022-05-03 08:31:48 +0000 UTC - event for smb-volume-tester-hp5hf: {default-scheduler } Scheduled: Successfully assigned smb-3274/smb-volume-tester-hp5hf to 2935k8s000
May  3 08:38:30.024: INFO: At 2022-05-03 08:33:51 +0000 UTC - event for smb-volume-tester-hp5hf: {kubelet 2935k8s000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-mnnhg]: timed out waiting for the condition
May  3 08:38:30.024: INFO: At 2022-05-03 08:38:24 +0000 UTC - event for smb-volume-tester-lmc75: {kubelet 2935k8s001} Killing: Stopping container volume-tester
May  3 08:38:30.055: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
May  3 08:38:30.055: INFO: 
May  3 08:38:30.090: INFO: 
Logging node info for node 2935k8s000
May  3 08:38:30.122: INFO: Node Info: &Node{ObjectMeta:{2935k8s000    0cc45ebd-421a-4b2d-aadb-9ed0a8bd23ee 5347 0 2022-05-03 07:46:55 +0000 UTC <nil> <nil> map[agentpool:windowspool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:windows failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:2935k8s000 kubernetes.io/os:windows kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 node.kubernetes.io/windows-build:10.0.17763 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:0] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kube-controller-manager Update v1 2022-05-03 07:46:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet.exe Update v1 2022-05-03 07:46:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:node.kubernetes.io/windows-build":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:46:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kubelet.exe Update v1 2022-05-03 08:08:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/2935k8s000,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{8589463552 0} {<nil>} 8388148Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{6441979904 0} {<nil>} 6290996Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:34:01 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:34:01 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:34:01 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:34:01 +0000 UTC,LastTransitionTime:2022-05-03 07:47:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:2935k8s000,},NodeAddress{Type:InternalIP,Address:10.240.0.4,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2935k8s000,SystemUUID:8C6455CB-2E09-47C9-A468-8A3D4A6E7FD9,BootID:,KernelVersion:10.0.17763.1879,OSImage:Windows Server 2019 Datacenter,ContainerRuntimeVersion:docker://20.10.9,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:windows,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/windows/servercore@sha256:1bb6543a7fe87bb95d9adfa4c53ea3469c1c846ad63baf32729078062e8c1127 mcr.microsoft.com/windows/servercore:ltsc2019],SizeBytes:5265856881,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:5ecb6e4f2ad6e12e153c396c68c2e412fc802514444d7ec7808806a50a340fe7 k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:287591069,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:4fd21f36075b44d1a423dfb262ad79202ce54e95f5cbc4622a6c1c38ab287ad6 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.0],SizeBytes:275364068,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:273371876,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:92dd20279c1e36282bd52e3db2679297921d517ffe4e98025c0370ce25a03fac mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v1.2.1-alpha.1-windows-1809-amd64],SizeBytes:265664693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:d07f92ef2517c99cd011391ab1182c995cdd04ec01e47324abf07ea502476409 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.0.1-alpha.1-windows-1809-amd64],SizeBytes:264960693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e075a40446ece06b77075d247bbf027c039c26b956418b1c5b605330be1217f9 mcr.microsoft.com/oss/kubernetes/pause:1.4.1],SizeBytes:256028579,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 kubletwin/pause:latest mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:254580394,},ContainerImage{Names:[mcr.microsoft.com/windows/nanoserver@sha256:8746dc8b6726d53ba7176332580712c2c5622290b2b44345ded8e302356203ac mcr.microsoft.com/windows/nanoserver:1809],SizeBytes:251866052,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:38:30.122: INFO: 
Logging kubelet events for node 2935k8s000
May  3 08:38:30.157: INFO: 
Logging pods the kubelet thinks is on node 2935k8s000
May  3 08:38:30.225: INFO: csi-smb-node-win-fnfxc started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:38:30.225: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:38:30.225: INFO: 	Container node-driver-registrar ready: true, restart count 11
May  3 08:38:30.225: INFO: 	Container smb ready: true, restart count 0
May  3 08:38:30.457: INFO: 
Latency metrics for node 2935k8s000
May  3 08:38:30.457: INFO: 
Logging node info for node 2935k8s001
May  3 08:38:30.490: INFO: Node Info: &Node{ObjectMeta:{2935k8s001    9931a1da-e618-4aa4-9401-7246e6c3b67b 5651 0 2022-05-03 07:47:00 +0000 UTC <nil> <nil> map[agentpool:windowspool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:windows failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:1 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:2935k8s001 kubernetes.io/os:windows kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 node.kubernetes.io/windows-build:10.0.17763 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:1] map[csi.volume.kubernetes.io/nodeid:{"smb.csi.k8s.io":"2935k8s001"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kube-controller-manager Update v1 2022-05-03 07:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet.exe Update v1 2022-05-03 07:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:node.kubernetes.io/windows-build":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:47:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kubelet.exe Update v1 2022-05-03 08:08:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/2935k8s001,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{8589463552 0} {<nil>} 8388148Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{6441979904 0} {<nil>} 6290996Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:37:07 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:37:07 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:37:07 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:37:07 +0000 UTC,LastTransitionTime:2022-05-03 07:47:10 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:2935k8s001,},NodeAddress{Type:InternalIP,Address:10.240.0.65,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2935k8s001,SystemUUID:CF6606F6-7E65-4189-B311-9E8E8C38EE4D,BootID:,KernelVersion:10.0.17763.1879,OSImage:Windows Server 2019 Datacenter,ContainerRuntimeVersion:docker://20.10.9,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:windows,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/windows/servercore@sha256:1bb6543a7fe87bb95d9adfa4c53ea3469c1c846ad63baf32729078062e8c1127 mcr.microsoft.com/windows/servercore:ltsc2019],SizeBytes:5265856881,},ContainerImage{Names:[e2eteam/busybox@sha256:240de552b18aa70ee4133cecca1171a43ea9cf0920acffb2d116cb8d8de9cddd e2eteam/busybox:1.29],SizeBytes:413218463,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:5ecb6e4f2ad6e12e153c396c68c2e412fc802514444d7ec7808806a50a340fe7 k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:287591069,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:4fd21f36075b44d1a423dfb262ad79202ce54e95f5cbc4622a6c1c38ab287ad6 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.0],SizeBytes:275364068,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:273371876,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:92dd20279c1e36282bd52e3db2679297921d517ffe4e98025c0370ce25a03fac mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v1.2.1-alpha.1-windows-1809-amd64],SizeBytes:265664693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:d07f92ef2517c99cd011391ab1182c995cdd04ec01e47324abf07ea502476409 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.0.1-alpha.1-windows-1809-amd64],SizeBytes:264960693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e075a40446ece06b77075d247bbf027c039c26b956418b1c5b605330be1217f9 mcr.microsoft.com/oss/kubernetes/pause:1.4.1],SizeBytes:256028579,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 kubletwin/pause:latest mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:254580394,},ContainerImage{Names:[mcr.microsoft.com/windows/nanoserver@sha256:8746dc8b6726d53ba7176332580712c2c5622290b2b44345ded8e302356203ac mcr.microsoft.com/windows/nanoserver:1809],SizeBytes:251866052,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:38:30.490: INFO: 
Logging kubelet events for node 2935k8s001
May  3 08:38:30.525: INFO: 
Logging pods the kubelet thinks is on node 2935k8s001
May  3 08:38:30.572: INFO: csi-smb-node-win-8l4q4 started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:38:30.572: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:38:30.572: INFO: 	Container node-driver-registrar ready: true, restart count 0
May  3 08:38:30.572: INFO: 	Container smb ready: true, restart count 0
May  3 08:38:30.770: INFO: 
Latency metrics for node 2935k8s001
May  3 08:38:30.770: INFO: 
Logging node info for node k8s-master-29351631-0
May  3 08:38:30.803: INFO: Node Info: &Node{ObjectMeta:{k8s-master-29351631-0    4e179e14-aca2-4f8b-b6a0-9e9da86d6233 5344 0 2022-05-03 07:44:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:master kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-master-29351631-0 kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/master: node.kubernetes.io/exclude-disruption:true node.kubernetes.io/exclude-from-external-load-balancers:true node.kubernetes.io/instance-type:Standard_D2_v3 topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:0] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-05-03 07:44:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/exclude-disruption":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:44:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-05-03 07:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet Update v1 2022-05-03 07:45:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/k8s-master-29351631-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344731648 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558299648 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:33:59 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:33:59 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:33:59 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:33:59 +0000 UTC,LastTransitionTime:2022-05-03 07:45:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-master-29351631-0,},NodeAddress{Type:InternalIP,Address:10.255.255.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:05c3b26f052f42068b637e8504dfd0ed,SystemUUID:60ccb959-1914-ba42-a701-7a4085e0a4b4,BootID:e4b8bab9-5341-4ef3-b060-29ab390539e4,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:3c8fd9a8710f05be99405947ada32cd9e4a6657349e32873172da61dd2ba56ba k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:190805704,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver@sha256:808a6792d8bbfbe22951ac274b3931c2b4484e0e6e09cdf93c8c887b25257c76 mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.8],SizeBytes:128319182,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager@sha256:298409ea9430b099d031ee92f9a128ab85d9531977ac0a30f651eb769f66dd7d mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.8],SizeBytes:122007567,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:29c53a2a5c90a860e9ab8dcff1f4bc93b70781958de71611a711fcaaea3b09ef mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.8],SizeBytes:103650551,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57729376,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[andyzhangx/samba@sha256:1c268eae7ec1eb9fbbbd4f0aab3ee41c7a4c4ecb0313c498cb11cfdcc762ad3e andyzhangx/samba:win-fix],SizeBytes:54262056,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:cf4a41eb7c3f844556059845a9d486e8b1e407a61285d70093fa1056d4bbb91b mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.8],SizeBytes:52670775,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17605286,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:38:30.804: INFO: 
Logging kubelet events for node k8s-master-29351631-0
May  3 08:38:30.842: INFO: 
Logging pods the kubelet thinks is on node k8s-master-29351631-0
May  3 08:38:30.921: INFO: kube-addon-manager-k8s-master-29351631-0 started at 2022-05-03 07:45:39 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container kube-addon-manager ready: true, restart count 0
May  3 08:38:30.921: INFO: kube-controller-manager-k8s-master-29351631-0 started at 2022-05-03 07:44:15 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container kube-controller-manager ready: true, restart count 0
May  3 08:38:30.921: INFO: coredns-745f545b98-rnmxq started at 2022-05-03 07:45:46 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container coredns ready: true, restart count 0
May  3 08:38:30.921: INFO: azure-ip-masq-agent-zfzpr started at 2022-05-03 07:45:51 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container azure-ip-masq-agent ready: true, restart count 0
May  3 08:38:30.921: INFO: metrics-server-7bf8c745cc-2vkbr started at 2022-05-03 07:46:00 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container metrics-server ready: true, restart count 0
May  3 08:38:30.921: INFO: smb-server-589b6679cb-hnz5f started at 2022-05-03 07:48:31 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container smb-server ready: true, restart count 0
May  3 08:38:30.921: INFO: csi-smb-controller-856896f755-fnl5f started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container csi-provisioner ready: true, restart count 0
May  3 08:38:30.921: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:38:30.921: INFO: 	Container smb ready: true, restart count 0
May  3 08:38:30.921: INFO: kube-scheduler-k8s-master-29351631-0 started at 2022-05-03 07:44:14 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container kube-scheduler ready: true, restart count 0
May  3 08:38:30.921: INFO: kube-apiserver-k8s-master-29351631-0 started at 2022-05-03 07:44:15 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container kube-apiserver ready: true, restart count 0
May  3 08:38:30.921: INFO: coredns-autoscaler-66d6fbf7c7-2t68r started at 2022-05-03 07:45:46 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container autoscaler ready: true, restart count 0
May  3 08:38:30.921: INFO: kube-proxy-hqv66 started at 2022-05-03 07:45:51 +0000 UTC (0+1 container statuses recorded)
May  3 08:38:30.921: INFO: 	Container kube-proxy ready: true, restart count 0
May  3 08:38:31.063: INFO: 
Latency metrics for node k8s-master-29351631-0
May  3 08:38:31.063: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
�[1mSTEP�[0m: Destroying namespace "smb-3274" for this suite.

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

SMB CSI Driver End-to-End Tests Dynamic Provisioning should create a volume on demand with mount options [smb.csi.k8s.io] [Windows] 15m46s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=SMB\sCSI\sDriver\sEnd\-to\-End\sTests\sDynamic\sProvisioning\sshould\screate\sa\svolume\son\sdemand\swith\smount\soptions\s\[smb\.csi\.k8s\.io\]\s\[Windows\]$'

/home/prow/go/src/sigs.k8s.io/csi-driver-smb/test/e2e/dynamic_provisioning_test.go:98
Unexpected error:
    <*errors.errorString | 0xc000119370>: {
        s: "Gave up after waiting 15m0s for pod \"smb-volume-tester-s5rtn\" to be \"Succeeded or Failed\"",
    }
    Gave up after waiting 15m0s for pod "smb-volume-tester-s5rtn" to be "Succeeded or Failed"
occurred
/home/prow/go/src/sigs.k8s.io/csi-driver-smb/test/e2e/testsuites/testsuites.go:527
				�[1mSTEP�[0m: Creating a kubernetes client
May  3 08:08:58.132: INFO: >>> kubeConfig: /root/tmp3374271991/kubeconfig/kubeconfig.eastus.json
�[1mSTEP�[0m: Building a namespace api object, basename smb
�[1mSTEP�[0m: Binding the e2e-test-privileged-psp PodSecurityPolicy to the default service account in smb-1318
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
�[1mSTEP�[0m: setting up the StorageClass
�[1mSTEP�[0m: creating a StorageClass 
�[1mSTEP�[0m: setting up the PVC and PV
�[1mSTEP�[0m: creating a PVC
�[1mSTEP�[0m: waiting for PVC to be in phase "Bound"
May  3 08:08:58.910: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-z2p5r] to have phase Bound
May  3 08:08:58.942: INFO: PersistentVolumeClaim pvc-z2p5r found but phase is Pending instead of Bound.
May  3 08:09:00.975: INFO: PersistentVolumeClaim pvc-z2p5r found and phase=Bound (2.06484117s)
�[1mSTEP�[0m: checking the PVC
�[1mSTEP�[0m: validating provisioned PV
�[1mSTEP�[0m: checking the PV
�[1mSTEP�[0m: deploying the pod
�[1mSTEP�[0m: checking that the pods command exits with no error
May  3 08:09:01.088: INFO: Waiting up to 15m0s for pod "smb-volume-tester-s5rtn" in namespace "smb-1318" to be "Succeeded or Failed"
May  3 08:09:01.139: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 50.95518ms
May  3 08:09:03.174: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.085915999s
May  3 08:09:05.207: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.119684312s
May  3 08:09:07.241: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.153157533s
May  3 08:09:09.281: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.193818674s
May  3 08:09:11.315: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.227262783s
May  3 08:09:13.349: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12.261429988s
May  3 08:09:15.383: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14.295421655s
May  3 08:09:17.419: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 16.331087383s
May  3 08:09:19.454: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 18.366620522s
May  3 08:09:21.488: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 20.400202698s
May  3 08:09:23.524: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 22.436384356s
May  3 08:09:25.559: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 24.471375505s
May  3 08:09:27.594: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 26.506041223s
May  3 08:09:29.629: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 28.541189281s
May  3 08:09:31.663: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 30.575220637s
May  3 08:09:33.698: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 32.610424277s
May  3 08:09:35.734: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 34.646136447s
May  3 08:09:37.768: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 36.680217611s
May  3 08:09:39.802: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 38.714715543s
May  3 08:09:41.836: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 40.74786087s
May  3 08:09:43.870: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 42.782228749s
May  3 08:09:45.904: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 44.816709731s
May  3 08:09:47.939: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 46.851132345s
May  3 08:09:49.973: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 48.884997922s
May  3 08:09:52.007: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 50.919073207s
May  3 08:09:54.041: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 52.953118448s
May  3 08:09:56.075: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 54.987529665s
May  3 08:09:58.109: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 57.021254078s
May  3 08:10:00.145: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 59.057693256s
May  3 08:10:02.179: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m1.091013131s
May  3 08:10:04.213: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.125207948s
May  3 08:10:06.247: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m5.159281379s
May  3 08:10:08.282: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m7.194535644s
May  3 08:10:10.319: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m9.231435243s
May  3 08:10:12.358: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m11.269864614s
May  3 08:10:14.390: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m13.302412813s
May  3 08:10:16.425: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m15.337430233s
May  3 08:10:18.459: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m17.371419872s
May  3 08:10:20.493: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m19.405118951s
May  3 08:10:22.527: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m21.438894257s
May  3 08:10:24.560: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m23.472218899s
May  3 08:10:26.596: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m25.508228488s
May  3 08:10:28.628: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m27.540676483s
May  3 08:10:30.661: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m29.573408355s
May  3 08:10:32.696: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m31.607980633s
May  3 08:10:34.729: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m33.641675093s
May  3 08:10:36.763: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m35.675041871s
May  3 08:10:38.797: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m37.70913827s
May  3 08:10:40.831: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m39.743370876s
May  3 08:10:42.866: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m41.778853037s
May  3 08:10:44.901: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m43.813324778s
May  3 08:10:46.934: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m45.84669016s
May  3 08:10:48.968: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m47.880705121s
May  3 08:10:51.004: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m49.915886273s
May  3 08:10:53.038: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m51.950309059s
May  3 08:10:55.072: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m53.984400474s
May  3 08:10:57.106: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.018379672s
May  3 08:10:59.140: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.052494883s
May  3 08:11:01.174: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.085955959s
May  3 08:11:03.208: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.120333473s
May  3 08:11:05.246: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.158239698s
May  3 08:11:07.280: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.192402463s
May  3 08:11:09.314: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.225872028s
May  3 08:11:11.347: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.259543465s
May  3 08:11:13.381: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.293052472s
May  3 08:11:15.414: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.326607109s
May  3 08:11:17.449: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.361041914s
May  3 08:11:19.482: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.394465957s
May  3 08:11:21.515: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.427646745s
May  3 08:11:23.550: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.462505718s
May  3 08:11:25.584: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.496440634s
May  3 08:11:27.619: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.530999985s
May  3 08:11:29.653: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.565568745s
May  3 08:11:31.689: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.601703068s
May  3 08:11:33.723: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.635406058s
May  3 08:11:35.757: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.669230944s
May  3 08:11:37.791: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.703826124s
May  3 08:11:39.826: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.738625523s
May  3 08:11:41.859: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.771713358s
May  3 08:11:43.893: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.805599174s
May  3 08:11:45.927: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.839003105s
May  3 08:11:47.961: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.87315475s
May  3 08:11:49.998: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.910658248s
May  3 08:11:52.033: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.945579001s
May  3 08:11:54.078: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.990701934s
May  3 08:11:56.112: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m55.024710242s
May  3 08:11:58.147: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m57.058968256s
May  3 08:12:00.181: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 2m59.092937673s
May  3 08:12:02.215: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m1.127014385s
May  3 08:12:04.249: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m3.161807893s
May  3 08:12:06.285: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m5.197662264s
May  3 08:12:08.320: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m7.232170239s
May  3 08:12:10.354: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m9.266304867s
May  3 08:12:12.392: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m11.304492345s
May  3 08:12:14.428: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m13.340077057s
May  3 08:12:16.463: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m15.375289563s
May  3 08:12:18.498: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m17.410224534s
May  3 08:12:20.531: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m19.443402971s
May  3 08:12:22.567: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m21.478999861s
May  3 08:12:24.600: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m23.512224941s
May  3 08:12:26.636: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m25.547877077s
May  3 08:12:28.674: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m27.586411288s
May  3 08:12:30.709: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m29.621157584s
May  3 08:12:32.743: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m31.655179838s
May  3 08:12:34.777: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m33.688956092s
May  3 08:12:36.811: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m35.723440328s
May  3 08:12:38.845: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m37.75719202s
May  3 08:12:40.880: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m39.791865292s
May  3 08:12:42.914: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m41.826814879s
May  3 08:12:44.949: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m43.861130298s
May  3 08:12:46.984: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m45.896751776s
May  3 08:12:49.017: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m47.929727112s
May  3 08:12:51.051: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m49.963354285s
May  3 08:12:53.086: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m51.998454468s
May  3 08:12:55.121: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.033398085s
May  3 08:12:57.166: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.077907156s
May  3 08:12:59.199: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.111371178s
May  3 08:13:01.233: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.14515505s
May  3 08:13:03.344: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.256212591s
May  3 08:13:05.382: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.294727141s
May  3 08:13:07.416: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.328257276s
May  3 08:13:09.448: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.360699365s
May  3 08:13:11.481: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.393411406s
May  3 08:13:13.515: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.427325091s
May  3 08:13:15.548: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.460538639s
May  3 08:13:17.582: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.494004807s
May  3 08:13:19.615: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.527542569s
May  3 08:13:21.649: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.561349288s
May  3 08:13:23.683: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.595381589s
May  3 08:13:25.716: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.628224626s
May  3 08:13:27.750: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.662565044s
May  3 08:13:29.784: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.696284815s
May  3 08:13:31.818: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.7300046s
May  3 08:13:33.852: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.764177941s
May  3 08:13:35.886: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.798727797s
May  3 08:13:37.921: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.832936411s
May  3 08:13:39.956: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.867905813s
May  3 08:13:41.990: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.902553331s
May  3 08:13:44.024: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.936538553s
May  3 08:13:46.058: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.970314418s
May  3 08:13:48.092: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m47.004389974s
May  3 08:13:50.127: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m49.038914628s
May  3 08:13:52.162: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m51.074360725s
May  3 08:13:54.197: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m53.10921073s
May  3 08:13:56.234: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m55.146748375s
May  3 08:13:58.270: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m57.181876578s
May  3 08:14:00.304: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 4m59.216656754s
May  3 08:14:02.340: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m1.252183628s
May  3 08:14:04.374: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m3.286381518s
May  3 08:14:06.407: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m5.319417822s
May  3 08:14:08.442: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m7.354024816s
May  3 08:14:10.476: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m9.388405524s
May  3 08:14:12.513: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m11.425185342s
May  3 08:14:14.548: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m13.460683072s
May  3 08:14:16.588: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m15.500241348s
May  3 08:14:18.623: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m17.535324332s
May  3 08:14:20.657: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m19.569543189s
May  3 08:14:22.692: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m21.60426467s
May  3 08:14:24.726: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m23.638386754s
May  3 08:14:26.759: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m25.671648097s
May  3 08:14:28.793: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m27.704918851s
May  3 08:14:30.828: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m29.740347555s
May  3 08:14:32.862: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m31.774415111s
May  3 08:14:34.896: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m33.808380879s
May  3 08:14:36.930: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m35.842661818s
May  3 08:14:38.965: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m37.877821833s
May  3 08:14:41.011: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m39.92304224s
May  3 08:14:43.045: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m41.957195507s
May  3 08:14:45.079: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m43.991733917s
May  3 08:14:47.115: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m46.026858251s
May  3 08:14:49.149: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m48.061130098s
May  3 08:14:51.183: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m50.095673907s
May  3 08:14:53.218: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m52.130698459s
May  3 08:14:55.253: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m54.16500621s
May  3 08:14:57.287: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m56.199094845s
May  3 08:14:59.320: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 5m58.232370793s
May  3 08:15:01.354: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m0.266390313s
May  3 08:15:03.389: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m2.301441422s
May  3 08:15:05.424: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m4.336664377s
May  3 08:15:07.460: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m6.372508058s
May  3 08:15:09.495: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m8.407265364s
May  3 08:15:11.529: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m10.44111646s
May  3 08:15:13.562: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m12.474346053s
May  3 08:15:15.597: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m14.509135465s
May  3 08:15:17.632: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m16.544391709s
May  3 08:15:19.665: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m18.577605364s
May  3 08:15:21.699: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m20.611157739s
May  3 08:15:23.732: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m22.644696478s
May  3 08:15:25.767: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m24.678927209s
May  3 08:15:27.800: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m26.712563387s
May  3 08:15:29.835: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m28.747647577s
May  3 08:15:31.895: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m30.80712081s
May  3 08:15:33.929: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m32.84182591s
May  3 08:15:35.964: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m34.876197356s
May  3 08:15:37.999: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m36.911217716s
May  3 08:15:40.034: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m38.946041866s
May  3 08:15:42.068: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m40.979931127s
May  3 08:15:44.102: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m43.014080185s
May  3 08:15:46.135: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m45.047820502s
May  3 08:15:48.172: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m47.084402963s
May  3 08:15:50.220: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m49.13242962s
May  3 08:15:52.255: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m51.167622833s
May  3 08:15:54.291: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m53.203138004s
May  3 08:15:56.324: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m55.23600272s
May  3 08:15:58.360: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m57.272851784s
May  3 08:16:00.399: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 6m59.311048269s
May  3 08:16:02.433: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m1.345407828s
May  3 08:16:04.467: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m3.379768264s
May  3 08:16:06.501: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m5.413076904s
May  3 08:16:08.542: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m7.454477655s
May  3 08:16:10.594: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m9.506298899s
May  3 08:16:12.633: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m11.545602989s
May  3 08:16:14.668: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m13.580696602s
May  3 08:16:16.704: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m15.616716241s
May  3 08:16:18.737: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m17.649802878s
May  3 08:16:20.770: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m19.682435924s
May  3 08:16:22.804: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m21.715899163s
May  3 08:16:24.838: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m23.750047219s
May  3 08:16:26.881: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m25.79356562s
May  3 08:16:28.914: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m27.826140406s
May  3 08:16:30.948: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m29.860082955s
May  3 08:16:32.982: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m31.894206541s
May  3 08:16:35.015: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m33.927483531s
May  3 08:16:37.048: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m35.960788222s
May  3 08:16:39.082: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m37.994080709s
May  3 08:16:41.115: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m40.027249949s
May  3 08:16:43.160: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m42.07235838s
May  3 08:16:45.195: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m44.107484982s
May  3 08:16:47.229: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m46.140885888s
May  3 08:16:49.262: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m48.174478227s
May  3 08:16:51.302: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m50.214621358s
May  3 08:16:53.337: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m52.249001583s
May  3 08:16:55.371: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m54.283154315s
May  3 08:16:57.405: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m56.317621913s
May  3 08:16:59.441: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 7m58.353291017s
May  3 08:17:01.474: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m0.386424566s
May  3 08:17:03.508: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m2.420673488s
May  3 08:17:05.542: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m4.454840922s
May  3 08:17:07.577: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m6.489126019s
May  3 08:17:09.611: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m8.52325217s
May  3 08:17:11.645: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m10.557537445s
May  3 08:17:13.682: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m12.594830514s
May  3 08:17:15.718: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m14.630819045s
May  3 08:17:17.753: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m16.665174235s
May  3 08:17:19.786: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m18.698348222s
May  3 08:17:21.821: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m20.733214675s
May  3 08:17:23.863: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m22.775220322s
May  3 08:17:25.897: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m24.80930462s
May  3 08:17:27.932: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m26.844688759s
May  3 08:17:29.967: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m28.879821339s
May  3 08:17:32.002: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m30.913908377s
May  3 08:17:34.035: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m32.947028684s
May  3 08:17:36.069: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m34.981367244s
May  3 08:17:38.104: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m37.016757421s
May  3 08:17:40.138: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m39.050649677s
May  3 08:17:42.174: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m41.086522086s
May  3 08:17:44.208: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m43.120267903s
May  3 08:17:46.242: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m45.154253319s
May  3 08:17:48.275: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m47.187601398s
May  3 08:17:50.313: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m49.225379362s
May  3 08:17:52.348: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m51.260022272s
May  3 08:17:54.386: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m53.298518503s
May  3 08:17:56.421: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m55.333194175s
May  3 08:17:58.457: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m57.368970501s
May  3 08:18:00.493: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 8m59.405178143s
May  3 08:18:02.528: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m1.440202786s
May  3 08:18:04.564: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m3.47652782s
May  3 08:18:06.599: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m5.511544751s
May  3 08:18:08.636: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m7.54803085s
May  3 08:18:10.670: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m9.582574912s
May  3 08:18:12.706: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m11.618108722s
May  3 08:18:14.739: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m13.65127809s
May  3 08:18:16.773: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m15.685562927s
May  3 08:18:18.808: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m17.720220891s
May  3 08:18:20.842: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m19.753971182s
May  3 08:18:22.878: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m21.790324918s
May  3 08:18:24.913: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m23.824986101s
May  3 08:18:26.948: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m25.860267767s
May  3 08:18:28.985: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m27.896903487s
May  3 08:18:31.018: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m29.930321928s
May  3 08:18:33.053: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m31.965032797s
May  3 08:18:35.091: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m34.003654425s
May  3 08:18:37.126: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m36.038170256s
May  3 08:18:39.163: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m38.075209372s
May  3 08:18:41.197: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m40.109656711s
May  3 08:18:43.234: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m42.145980052s
May  3 08:18:45.268: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m44.180143638s
May  3 08:18:47.302: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m46.214358083s
May  3 08:18:49.336: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m48.248740879s
May  3 08:18:51.379: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m50.291434208s
May  3 08:18:53.414: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m52.326460346s
May  3 08:18:55.459: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m54.371638515s
May  3 08:18:57.492: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m56.404463601s
May  3 08:18:59.526: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 9m58.43808992s
May  3 08:19:01.559: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m0.471253424s
May  3 08:19:03.592: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m2.504734166s
May  3 08:19:05.626: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m4.538304822s
May  3 08:19:07.664: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m6.57613066s
May  3 08:19:09.698: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m8.610607732s
May  3 08:19:11.732: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m10.644551475s
May  3 08:19:13.765: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m12.677694013s
May  3 08:19:15.798: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m14.710779147s
May  3 08:19:17.832: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m16.744423526s
May  3 08:19:19.871: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m18.783751212s
May  3 08:19:21.906: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m20.818695538s
May  3 08:19:23.941: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m22.853447347s
May  3 08:19:25.981: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m24.893761893s
May  3 08:19:28.016: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m26.928417018s
May  3 08:19:30.050: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m28.962657422s
May  3 08:19:32.085: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m30.997805782s
May  3 08:19:34.120: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m33.032242735s
May  3 08:19:36.155: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m35.067173948s
May  3 08:19:38.188: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m37.100625718s
May  3 08:19:40.223: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m39.135035937s
May  3 08:19:42.258: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m41.170248798s
May  3 08:19:44.292: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m43.204778315s
May  3 08:19:46.327: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m45.239406532s
May  3 08:19:48.362: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m47.273887024s
May  3 08:19:50.397: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m49.309113892s
May  3 08:19:52.430: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m51.342623802s
May  3 08:19:54.465: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m53.377612602s
May  3 08:19:56.500: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m55.412614821s
May  3 08:19:58.535: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m57.447740909s
May  3 08:20:00.572: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 10m59.484240557s
May  3 08:20:02.606: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m1.518618966s
May  3 08:20:04.640: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m3.552298217s
May  3 08:20:06.679: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m5.591796864s
May  3 08:20:08.714: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m7.62682926s
May  3 08:20:10.747: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m9.659695128s
May  3 08:20:12.784: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m11.696171452s
May  3 08:20:14.819: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m13.73096299s
May  3 08:20:16.854: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m15.765944876s
May  3 08:20:18.889: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m17.801632377s
May  3 08:20:20.923: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m19.835381634s
May  3 08:20:22.957: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m21.869696225s
May  3 08:20:24.992: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m23.90404473s
May  3 08:20:27.026: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m25.938673055s
May  3 08:20:29.061: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m27.973455936s
May  3 08:20:31.095: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m30.007110978s
May  3 08:20:33.129: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m32.041660188s
May  3 08:20:35.164: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m34.075972275s
May  3 08:20:37.199: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m36.111338182s
May  3 08:20:39.233: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m38.14574086s
May  3 08:20:41.267: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m40.179533243s
May  3 08:20:43.303: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m42.215196304s
May  3 08:20:45.337: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m44.249610936s
May  3 08:20:47.372: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m46.284168304s
May  3 08:20:49.407: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m48.319808826s
May  3 08:20:51.442: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m50.354567861s
May  3 08:20:53.477: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m52.389831728s
May  3 08:20:55.512: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m54.424429168s
May  3 08:20:57.546: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m56.458719929s
May  3 08:20:59.582: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 11m58.494167016s
May  3 08:21:01.615: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m0.527144856s
May  3 08:21:03.649: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m2.561746855s
May  3 08:21:05.683: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m4.595196356s
May  3 08:21:07.717: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m6.629737743s
May  3 08:21:09.752: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m8.664385596s
May  3 08:21:11.786: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m10.697995872s
May  3 08:21:13.820: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m12.73205816s
May  3 08:21:15.855: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m14.767546578s
May  3 08:21:17.890: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m16.802079923s
May  3 08:21:19.923: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m18.835384444s
May  3 08:21:21.957: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m20.86965023s
May  3 08:21:23.991: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m22.903060836s
May  3 08:21:26.024: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m24.936496672s
May  3 08:21:28.059: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m26.971195526s
May  3 08:21:30.096: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m29.008455771s
May  3 08:21:32.131: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m31.043417694s
May  3 08:21:34.165: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m33.07760097s
May  3 08:21:36.199: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m35.111598335s
May  3 08:21:38.235: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m37.147293862s
May  3 08:21:40.270: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m39.182135695s
May  3 08:21:42.306: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m41.218096916s
May  3 08:21:44.341: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m43.253348964s
May  3 08:21:46.377: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m45.289008967s
May  3 08:21:48.411: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m47.323251745s
May  3 08:21:50.444: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m49.3566555s
May  3 08:21:52.480: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m51.392586204s
May  3 08:21:54.514: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m53.426247451s
May  3 08:21:56.553: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m55.464872115s
May  3 08:21:58.586: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m57.498349596s
May  3 08:22:00.620: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 12m59.532035034s
May  3 08:22:02.654: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m1.566188347s
May  3 08:22:04.689: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m3.601372363s
May  3 08:22:06.723: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m5.634993427s
May  3 08:22:08.756: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m7.668126487s
May  3 08:22:10.790: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m9.702557307s
May  3 08:22:12.824: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m11.736815238s
May  3 08:22:14.860: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m13.772411998s
May  3 08:22:16.893: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m15.805782565s
May  3 08:22:18.928: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m17.840238526s
May  3 08:22:20.962: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m19.874540683s
May  3 08:22:23.006: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m21.917876633s
May  3 08:22:25.040: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m23.952793119s
May  3 08:22:27.075: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m25.987298692s
May  3 08:22:29.109: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m28.021715388s
May  3 08:22:31.143: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m30.055046029s
May  3 08:22:33.177: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m32.089717462s
May  3 08:22:35.212: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m34.124296967s
May  3 08:22:37.246: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m36.158192835s
May  3 08:22:39.281: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m38.19287745s
May  3 08:22:41.315: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m40.227809681s
May  3 08:22:43.352: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m42.264232797s
May  3 08:22:45.388: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m44.300093749s
May  3 08:22:47.422: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m46.334805489s
May  3 08:22:49.460: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m48.372035361s
May  3 08:22:51.494: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m50.406558012s
May  3 08:22:53.529: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m52.440972212s
May  3 08:22:55.563: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m54.47513072s
May  3 08:22:57.597: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m56.50961918s
May  3 08:22:59.632: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 13m58.544065897s
May  3 08:23:01.666: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m0.578608054s
May  3 08:23:03.700: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m2.612758783s
May  3 08:23:05.733: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m4.645750343s
May  3 08:23:07.767: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m6.679386187s
May  3 08:23:09.803: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m8.715361865s
May  3 08:23:11.839: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m10.75112965s
May  3 08:23:13.877: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m12.789578375s
May  3 08:23:15.911: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m14.823622392s
May  3 08:23:17.945: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m16.85762062s
May  3 08:23:19.980: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m18.892091214s
May  3 08:23:22.015: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m20.927608735s
May  3 08:23:24.049: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m22.96099898s
May  3 08:23:26.083: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m24.995391629s
May  3 08:23:28.118: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m27.03017257s
May  3 08:23:30.154: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m29.066217328s
May  3 08:23:32.188: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m31.100382869s
May  3 08:23:34.222: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m33.134678137s
May  3 08:23:36.256: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m35.168538198s
May  3 08:23:38.292: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m37.204209788s
May  3 08:23:40.325: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m39.237464998s
May  3 08:23:42.362: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m41.274761143s
May  3 08:23:44.397: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m43.30947784s
May  3 08:23:46.432: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m45.344434563s
May  3 08:23:48.467: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m47.379116224s
May  3 08:23:50.500: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m49.412394715s
May  3 08:23:52.535: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.447218389s
May  3 08:23:54.570: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.482167986s
May  3 08:23:56.603: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.515188074s
May  3 08:23:58.637: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.549505018s
May  3 08:24:00.671: INFO: Pod "smb-volume-tester-s5rtn": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.583069279s
May  3 08:24:02.672: INFO: deleting Pod "smb-1318"/"smb-volume-tester-s5rtn"
May  3 08:24:02.761: INFO: Error getting logs for pod smb-volume-tester-s5rtn: the server rejected our request for an unknown reason (get pods smb-volume-tester-s5rtn)
�[1mSTEP�[0m: Deleting pod smb-volume-tester-s5rtn in namespace smb-1318
May  3 08:24:02.805: INFO: deleting PVC "smb-1318"/"pvc-z2p5r"
May  3 08:24:02.805: INFO: Deleting PersistentVolumeClaim "pvc-z2p5r"
�[1mSTEP�[0m: waiting for claim's PV "pvc-8c6f3bb8-4afd-42c3-b688-03955a380596" to be deleted
May  3 08:24:02.926: INFO: Waiting up to 10m0s for PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 to get deleted
May  3 08:24:02.963: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (37.43361ms)
May  3 08:24:07.999: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (5.072774053s)
May  3 08:24:13.038: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (10.112119317s)
May  3 08:24:18.071: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (15.145279352s)
May  3 08:24:23.106: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (20.180181874s)
May  3 08:24:28.139: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (25.213383652s)
May  3 08:24:33.177: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (30.250999086s)
May  3 08:24:38.212: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 found and phase=Bound (35.286008631s)
May  3 08:24:43.249: INFO: PersistentVolume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596 was removed
May  3 08:24:43.249: INFO: Waiting up to 5m0s for PersistentVolumeClaim smb-1318 to be removed
May  3 08:24:43.281: INFO: Claim "smb-1318" in namespace "pvc-z2p5r" doesn't exist in the system
May  3 08:24:43.281: INFO: deleting StorageClass smb-1318-smb.csi.k8s.io-dynamic-sc-zgksr
�[1mSTEP�[0m: Collecting events from namespace "smb-1318".
�[1mSTEP�[0m: Found 7 events.
May  3 08:24:43.351: INFO: At 2022-05-03 08:08:58 +0000 UTC - event for pvc-z2p5r: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } Provisioning: External provisioner is provisioning volume for claim "smb-1318/pvc-z2p5r"
May  3 08:24:43.351: INFO: At 2022-05-03 08:08:58 +0000 UTC - event for pvc-z2p5r: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "smb.csi.k8s.io" or manually created by system administrator
May  3 08:24:43.351: INFO: At 2022-05-03 08:08:59 +0000 UTC - event for pvc-z2p5r: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } ProvisioningSucceeded: Successfully provisioned volume pvc-8c6f3bb8-4afd-42c3-b688-03955a380596
May  3 08:24:43.351: INFO: At 2022-05-03 08:09:01 +0000 UTC - event for smb-volume-tester-s5rtn: {default-scheduler } Scheduled: Successfully assigned smb-1318/smb-volume-tester-s5rtn to 2935k8s000
May  3 08:24:43.351: INFO: At 2022-05-03 08:09:01 +0000 UTC - event for smb-volume-tester-s5rtn: {kubelet 2935k8s000} FailedMount: MountVolume.MountDevice failed for volume "pvc-8c6f3bb8-4afd-42c3-b688-03955a380596" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name smb.csi.k8s.io not found in the list of registered CSI drivers
May  3 08:24:43.351: INFO: At 2022-05-03 08:11:04 +0000 UTC - event for smb-volume-tester-s5rtn: {kubelet 2935k8s000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-sdchz]: timed out waiting for the condition
May  3 08:24:43.351: INFO: At 2022-05-03 08:13:22 +0000 UTC - event for smb-volume-tester-s5rtn: {kubelet 2935k8s000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[kube-api-access-sdchz test-volume-1]: timed out waiting for the condition
May  3 08:24:43.384: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
May  3 08:24:43.384: INFO: 
May  3 08:24:43.447: INFO: 
Logging node info for node 2935k8s000
May  3 08:24:43.481: INFO: Node Info: &Node{ObjectMeta:{2935k8s000    0cc45ebd-421a-4b2d-aadb-9ed0a8bd23ee 4240 0 2022-05-03 07:46:55 +0000 UTC <nil> <nil> map[agentpool:windowspool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:windows failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:2935k8s000 kubernetes.io/os:windows kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 node.kubernetes.io/windows-build:10.0.17763 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:0] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kube-controller-manager Update v1 2022-05-03 07:46:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet.exe Update v1 2022-05-03 07:46:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:node.kubernetes.io/windows-build":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:46:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kubelet.exe Update v1 2022-05-03 08:08:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/2935k8s000,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{8589463552 0} {<nil>} 8388148Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{6441979904 0} {<nil>} 6290996Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:24:00 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:24:00 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:24:00 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:24:00 +0000 UTC,LastTransitionTime:2022-05-03 07:47:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:2935k8s000,},NodeAddress{Type:InternalIP,Address:10.240.0.4,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2935k8s000,SystemUUID:8C6455CB-2E09-47C9-A468-8A3D4A6E7FD9,BootID:,KernelVersion:10.0.17763.1879,OSImage:Windows Server 2019 Datacenter,ContainerRuntimeVersion:docker://20.10.9,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:windows,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/windows/servercore@sha256:1bb6543a7fe87bb95d9adfa4c53ea3469c1c846ad63baf32729078062e8c1127 mcr.microsoft.com/windows/servercore:ltsc2019],SizeBytes:5265856881,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:5ecb6e4f2ad6e12e153c396c68c2e412fc802514444d7ec7808806a50a340fe7 k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:287591069,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:4fd21f36075b44d1a423dfb262ad79202ce54e95f5cbc4622a6c1c38ab287ad6 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.0],SizeBytes:275364068,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:273371876,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:92dd20279c1e36282bd52e3db2679297921d517ffe4e98025c0370ce25a03fac mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v1.2.1-alpha.1-windows-1809-amd64],SizeBytes:265664693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:d07f92ef2517c99cd011391ab1182c995cdd04ec01e47324abf07ea502476409 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.0.1-alpha.1-windows-1809-amd64],SizeBytes:264960693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e075a40446ece06b77075d247bbf027c039c26b956418b1c5b605330be1217f9 mcr.microsoft.com/oss/kubernetes/pause:1.4.1],SizeBytes:256028579,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 kubletwin/pause:latest mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:254580394,},ContainerImage{Names:[mcr.microsoft.com/windows/nanoserver@sha256:8746dc8b6726d53ba7176332580712c2c5622290b2b44345ded8e302356203ac mcr.microsoft.com/windows/nanoserver:1809],SizeBytes:251866052,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:24:43.482: INFO: 
Logging kubelet events for node 2935k8s000
May  3 08:24:43.516: INFO: 
Logging pods the kubelet thinks is on node 2935k8s000
May  3 08:24:43.556: INFO: csi-smb-node-win-fnfxc started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:24:43.556: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:24:43.556: INFO: 	Container node-driver-registrar ready: false, restart count 8
May  3 08:24:43.556: INFO: 	Container smb ready: true, restart count 0
May  3 08:24:43.776: INFO: 
Latency metrics for node 2935k8s000
May  3 08:24:43.776: INFO: 
Logging node info for node 2935k8s001
May  3 08:24:43.808: INFO: Node Info: &Node{ObjectMeta:{2935k8s001    9931a1da-e618-4aa4-9401-7246e6c3b67b 4254 0 2022-05-03 07:47:00 +0000 UTC <nil> <nil> map[agentpool:windowspool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:windows failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:1 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:2935k8s001 kubernetes.io/os:windows kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 node.kubernetes.io/windows-build:10.0.17763 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:1] map[csi.volume.kubernetes.io/nodeid:{"smb.csi.k8s.io":"2935k8s001"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kube-controller-manager Update v1 2022-05-03 07:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet.exe Update v1 2022-05-03 07:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:node.kubernetes.io/windows-build":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:47:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kubelet.exe Update v1 2022-05-03 08:08:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/2935k8s001,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{8589463552 0} {<nil>} 8388148Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{6441979904 0} {<nil>} 6290996Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:24:05 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:24:05 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:24:05 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:24:05 +0000 UTC,LastTransitionTime:2022-05-03 07:47:10 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:2935k8s001,},NodeAddress{Type:InternalIP,Address:10.240.0.65,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2935k8s001,SystemUUID:CF6606F6-7E65-4189-B311-9E8E8C38EE4D,BootID:,KernelVersion:10.0.17763.1879,OSImage:Windows Server 2019 Datacenter,ContainerRuntimeVersion:docker://20.10.9,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:windows,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/windows/servercore@sha256:1bb6543a7fe87bb95d9adfa4c53ea3469c1c846ad63baf32729078062e8c1127 mcr.microsoft.com/windows/servercore:ltsc2019],SizeBytes:5265856881,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:5ecb6e4f2ad6e12e153c396c68c2e412fc802514444d7ec7808806a50a340fe7 k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:287591069,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:4fd21f36075b44d1a423dfb262ad79202ce54e95f5cbc4622a6c1c38ab287ad6 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.0],SizeBytes:275364068,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:273371876,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:92dd20279c1e36282bd52e3db2679297921d517ffe4e98025c0370ce25a03fac mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v1.2.1-alpha.1-windows-1809-amd64],SizeBytes:265664693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:d07f92ef2517c99cd011391ab1182c995cdd04ec01e47324abf07ea502476409 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.0.1-alpha.1-windows-1809-amd64],SizeBytes:264960693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e075a40446ece06b77075d247bbf027c039c26b956418b1c5b605330be1217f9 mcr.microsoft.com/oss/kubernetes/pause:1.4.1],SizeBytes:256028579,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 kubletwin/pause:latest mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:254580394,},ContainerImage{Names:[mcr.microsoft.com/windows/nanoserver@sha256:8746dc8b6726d53ba7176332580712c2c5622290b2b44345ded8e302356203ac mcr.microsoft.com/windows/nanoserver:1809],SizeBytes:251866052,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:24:43.809: INFO: 
Logging kubelet events for node 2935k8s001
May  3 08:24:43.844: INFO: 
Logging pods the kubelet thinks is on node 2935k8s001
May  3 08:24:43.980: INFO: csi-smb-node-win-8l4q4 started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:24:43.980: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:24:43.980: INFO: 	Container node-driver-registrar ready: true, restart count 0
May  3 08:24:43.980: INFO: 	Container smb ready: true, restart count 0
May  3 08:24:44.213: INFO: 
Latency metrics for node 2935k8s001
May  3 08:24:44.213: INFO: 
Logging node info for node k8s-master-29351631-0
May  3 08:24:44.247: INFO: Node Info: &Node{ObjectMeta:{k8s-master-29351631-0    4e179e14-aca2-4f8b-b6a0-9e9da86d6233 4234 0 2022-05-03 07:44:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:master kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-master-29351631-0 kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/master: node.kubernetes.io/exclude-disruption:true node.kubernetes.io/exclude-from-external-load-balancers:true node.kubernetes.io/instance-type:Standard_D2_v3 topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:0] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-05-03 07:44:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/exclude-disruption":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:44:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-05-03 07:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet Update v1 2022-05-03 07:45:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/k8s-master-29351631-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344731648 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558299648 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:23:56 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:23:56 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:23:56 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:23:56 +0000 UTC,LastTransitionTime:2022-05-03 07:45:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-master-29351631-0,},NodeAddress{Type:InternalIP,Address:10.255.255.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:05c3b26f052f42068b637e8504dfd0ed,SystemUUID:60ccb959-1914-ba42-a701-7a4085e0a4b4,BootID:e4b8bab9-5341-4ef3-b060-29ab390539e4,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:3c8fd9a8710f05be99405947ada32cd9e4a6657349e32873172da61dd2ba56ba k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:190805704,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver@sha256:808a6792d8bbfbe22951ac274b3931c2b4484e0e6e09cdf93c8c887b25257c76 mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.8],SizeBytes:128319182,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager@sha256:298409ea9430b099d031ee92f9a128ab85d9531977ac0a30f651eb769f66dd7d mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.8],SizeBytes:122007567,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:29c53a2a5c90a860e9ab8dcff1f4bc93b70781958de71611a711fcaaea3b09ef mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.8],SizeBytes:103650551,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57729376,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[andyzhangx/samba@sha256:1c268eae7ec1eb9fbbbd4f0aab3ee41c7a4c4ecb0313c498cb11cfdcc762ad3e andyzhangx/samba:win-fix],SizeBytes:54262056,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:cf4a41eb7c3f844556059845a9d486e8b1e407a61285d70093fa1056d4bbb91b mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.8],SizeBytes:52670775,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17605286,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:24:44.247: INFO: 
Logging kubelet events for node k8s-master-29351631-0
May  3 08:24:44.282: INFO: 
Logging pods the kubelet thinks is on node k8s-master-29351631-0
May  3 08:24:44.360: INFO: metrics-server-7bf8c745cc-2vkbr started at 2022-05-03 07:46:00 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container metrics-server ready: true, restart count 0
May  3 08:24:44.360: INFO: smb-server-589b6679cb-hnz5f started at 2022-05-03 07:48:31 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container smb-server ready: true, restart count 0
May  3 08:24:44.360: INFO: csi-smb-controller-856896f755-fnl5f started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container csi-provisioner ready: true, restart count 0
May  3 08:24:44.360: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:24:44.360: INFO: 	Container smb ready: true, restart count 0
May  3 08:24:44.360: INFO: kube-scheduler-k8s-master-29351631-0 started at 2022-05-03 07:44:14 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container kube-scheduler ready: true, restart count 0
May  3 08:24:44.360: INFO: kube-apiserver-k8s-master-29351631-0 started at 2022-05-03 07:44:15 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container kube-apiserver ready: true, restart count 0
May  3 08:24:44.360: INFO: coredns-autoscaler-66d6fbf7c7-2t68r started at 2022-05-03 07:45:46 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container autoscaler ready: true, restart count 0
May  3 08:24:44.360: INFO: kube-proxy-hqv66 started at 2022-05-03 07:45:51 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container kube-proxy ready: true, restart count 0
May  3 08:24:44.360: INFO: kube-addon-manager-k8s-master-29351631-0 started at 2022-05-03 07:45:39 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container kube-addon-manager ready: true, restart count 0
May  3 08:24:44.360: INFO: kube-controller-manager-k8s-master-29351631-0 started at 2022-05-03 07:44:15 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container kube-controller-manager ready: true, restart count 0
May  3 08:24:44.360: INFO: coredns-745f545b98-rnmxq started at 2022-05-03 07:45:46 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container coredns ready: true, restart count 0
May  3 08:24:44.360: INFO: azure-ip-masq-agent-zfzpr started at 2022-05-03 07:45:51 +0000 UTC (0+1 container statuses recorded)
May  3 08:24:44.360: INFO: 	Container azure-ip-masq-agent ready: true, restart count 0
May  3 08:24:44.508: INFO: 
Latency metrics for node k8s-master-29351631-0
May  3 08:24:44.509: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
�[1mSTEP�[0m: Destroying namespace "smb-1318" for this suite.

				Click to see stdout/stderrfrom junit_01.xml

Find smb-volume-tester-s5rtn mentions in log files | View test history on testgrid

SMB CSI Driver End-to-End Tests Dynamic Provisioning should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [smb.csi.k8s.io] [Windows] 6m40s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=SMB\sCSI\sDriver\sEnd\-to\-End\sTests\sDynamic\sProvisioning\sshould\screate\smultiple\sPV\sobjects\,\sbind\sto\sPVCs\sand\sattach\sall\sto\sdifferent\spods\son\sthe\ssame\snode\s\[smb\.csi\.k8s\.io\]\s\[Windows\]$'

/home/prow/go/src/sigs.k8s.io/csi-driver-smb/test/e2e/dynamic_provisioning_test.go:132
Unexpected error:
    <*errors.errorString | 0xc0002eabc0>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred
/home/prow/go/src/sigs.k8s.io/csi-driver-smb/test/e2e/testsuites/testsuites.go:532
				�[1mSTEP�[0m: Creating a kubernetes client
May  3 08:24:44.599: INFO: >>> kubeConfig: /root/tmp3374271991/kubeconfig/kubeconfig.eastus.json
�[1mSTEP�[0m: Building a namespace api object, basename smb
�[1mSTEP�[0m: Binding the e2e-test-privileged-psp PodSecurityPolicy to the default service account in smb-694
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
�[1mSTEP�[0m: setting up the StorageClass
�[1mSTEP�[0m: creating a StorageClass 
�[1mSTEP�[0m: setting up the PVC and PV
�[1mSTEP�[0m: creating a PVC
�[1mSTEP�[0m: waiting for PVC to be in phase "Bound"
May  3 08:24:45.569: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-vqbzh] to have phase Bound
May  3 08:24:45.611: INFO: PersistentVolumeClaim pvc-vqbzh found but phase is Pending instead of Bound.
May  3 08:24:47.646: INFO: PersistentVolumeClaim pvc-vqbzh found and phase=Bound (2.077209346s)
�[1mSTEP�[0m: checking the PVC
�[1mSTEP�[0m: validating provisioned PV
�[1mSTEP�[0m: checking the PV
�[1mSTEP�[0m: deploying the pod
�[1mSTEP�[0m: checking that the pod is running
May  3 08:29:47.875: INFO: deleting Pod "smb-694"/"smb-volume-tester-lwp56"
May  3 08:29:47.964: INFO: Error getting logs for pod smb-volume-tester-lwp56: the server rejected our request for an unknown reason (get pods smb-volume-tester-lwp56)
�[1mSTEP�[0m: Deleting pod smb-volume-tester-lwp56 in namespace smb-694
May  3 08:29:47.998: INFO: deleting PVC "smb-694"/"pvc-vqbzh"
May  3 08:29:47.998: INFO: Deleting PersistentVolumeClaim "pvc-vqbzh"
�[1mSTEP�[0m: waiting for claim's PV "pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f" to be deleted
May  3 08:29:48.104: INFO: Waiting up to 10m0s for PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f to get deleted
May  3 08:29:48.146: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (42.698946ms)
May  3 08:29:53.179: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (5.075850579s)
May  3 08:29:58.216: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (10.112631091s)
May  3 08:30:03.255: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (15.151303233s)
May  3 08:30:08.288: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (20.184747877s)
May  3 08:30:13.322: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (25.218739722s)
May  3 08:30:18.356: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (30.252391112s)
May  3 08:30:23.390: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (35.286141921s)
May  3 08:30:28.424: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (40.31993188s)
May  3 08:30:33.459: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (45.355302558s)
May  3 08:30:38.493: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (50.389102039s)
May  3 08:30:43.529: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (55.425508312s)
May  3 08:30:48.563: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (1m0.459483787s)
May  3 08:30:53.596: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (1m5.492900446s)
May  3 08:30:58.633: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (1m10.529288215s)
May  3 08:31:03.666: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (1m15.56225319s)
May  3 08:31:08.699: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (1m20.594912787s)
May  3 08:31:13.733: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (1m25.629192262s)
May  3 08:31:18.766: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f found and phase=Bound (1m30.662514055s)
May  3 08:31:23.802: INFO: PersistentVolume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f was removed
May  3 08:31:23.802: INFO: Waiting up to 5m0s for PersistentVolumeClaim smb-694 to be removed
May  3 08:31:23.834: INFO: Claim "smb-694" in namespace "pvc-vqbzh" doesn't exist in the system
May  3 08:31:23.835: INFO: deleting StorageClass smb-694-smb.csi.k8s.io-dynamic-sc-jnnj4
�[1mSTEP�[0m: Collecting events from namespace "smb-694".
�[1mSTEP�[0m: Found 6 events.
May  3 08:31:23.901: INFO: At 2022-05-03 08:24:45 +0000 UTC - event for pvc-vqbzh: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } Provisioning: External provisioner is provisioning volume for claim "smb-694/pvc-vqbzh"
May  3 08:31:23.901: INFO: At 2022-05-03 08:24:45 +0000 UTC - event for pvc-vqbzh: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "smb.csi.k8s.io" or manually created by system administrator
May  3 08:31:23.901: INFO: At 2022-05-03 08:24:45 +0000 UTC - event for pvc-vqbzh: {smb.csi.k8s.io_csi-smb-controller-856896f755-fnl5f_b17a5eb8-8b46-408c-9cdf-f670e040bd8b } ProvisioningSucceeded: Successfully provisioned volume pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f
May  3 08:31:23.901: INFO: At 2022-05-03 08:24:47 +0000 UTC - event for smb-volume-tester-lwp56: {default-scheduler } Scheduled: Successfully assigned smb-694/smb-volume-tester-lwp56 to 2935k8s000
May  3 08:31:23.901: INFO: At 2022-05-03 08:24:47 +0000 UTC - event for smb-volume-tester-lwp56: {kubelet 2935k8s000} FailedMount: MountVolume.MountDevice failed for volume "pvc-8b3b12f1-cb46-4a8e-8b8b-a50ddd9ded1f" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name smb.csi.k8s.io not found in the list of registered CSI drivers
May  3 08:31:23.901: INFO: At 2022-05-03 08:26:50 +0000 UTC - event for smb-volume-tester-lwp56: {kubelet 2935k8s000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 kube-api-access-gvrvp]: timed out waiting for the condition
May  3 08:31:23.941: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
May  3 08:31:23.941: INFO: 
May  3 08:31:24.004: INFO: 
Logging node info for node 2935k8s000
May  3 08:31:24.041: INFO: Node Info: &Node{ObjectMeta:{2935k8s000    0cc45ebd-421a-4b2d-aadb-9ed0a8bd23ee 4783 0 2022-05-03 07:46:55 +0000 UTC <nil> <nil> map[agentpool:windowspool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:windows failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:2935k8s000 kubernetes.io/os:windows kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 node.kubernetes.io/windows-build:10.0.17763 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:0] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kube-controller-manager Update v1 2022-05-03 07:46:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet.exe Update v1 2022-05-03 07:46:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:node.kubernetes.io/windows-build":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:46:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kubelet.exe Update v1 2022-05-03 08:08:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/2935k8s000,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{8589463552 0} {<nil>} 8388148Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{6441979904 0} {<nil>} 6290996Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:29:01 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:29:01 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:29:01 +0000 UTC,LastTransitionTime:2022-05-03 07:46:55 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:29:01 +0000 UTC,LastTransitionTime:2022-05-03 07:47:05 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:2935k8s000,},NodeAddress{Type:InternalIP,Address:10.240.0.4,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2935k8s000,SystemUUID:8C6455CB-2E09-47C9-A468-8A3D4A6E7FD9,BootID:,KernelVersion:10.0.17763.1879,OSImage:Windows Server 2019 Datacenter,ContainerRuntimeVersion:docker://20.10.9,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:windows,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/windows/servercore@sha256:1bb6543a7fe87bb95d9adfa4c53ea3469c1c846ad63baf32729078062e8c1127 mcr.microsoft.com/windows/servercore:ltsc2019],SizeBytes:5265856881,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:5ecb6e4f2ad6e12e153c396c68c2e412fc802514444d7ec7808806a50a340fe7 k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:287591069,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:4fd21f36075b44d1a423dfb262ad79202ce54e95f5cbc4622a6c1c38ab287ad6 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.0],SizeBytes:275364068,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:273371876,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:92dd20279c1e36282bd52e3db2679297921d517ffe4e98025c0370ce25a03fac mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v1.2.1-alpha.1-windows-1809-amd64],SizeBytes:265664693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:d07f92ef2517c99cd011391ab1182c995cdd04ec01e47324abf07ea502476409 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.0.1-alpha.1-windows-1809-amd64],SizeBytes:264960693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e075a40446ece06b77075d247bbf027c039c26b956418b1c5b605330be1217f9 mcr.microsoft.com/oss/kubernetes/pause:1.4.1],SizeBytes:256028579,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 kubletwin/pause:latest mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:254580394,},ContainerImage{Names:[mcr.microsoft.com/windows/nanoserver@sha256:8746dc8b6726d53ba7176332580712c2c5622290b2b44345ded8e302356203ac mcr.microsoft.com/windows/nanoserver:1809],SizeBytes:251866052,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:31:24.042: INFO: 
Logging kubelet events for node 2935k8s000
May  3 08:31:24.082: INFO: 
Logging pods the kubelet thinks is on node 2935k8s000
May  3 08:31:24.153: INFO: csi-smb-node-win-fnfxc started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:31:24.153: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:31:24.153: INFO: 	Container node-driver-registrar ready: true, restart count 10
May  3 08:31:24.153: INFO: 	Container smb ready: true, restart count 0
May  3 08:31:24.391: INFO: 
Latency metrics for node 2935k8s000
May  3 08:31:24.391: INFO: 
Logging node info for node 2935k8s001
May  3 08:31:24.423: INFO: Node Info: &Node{ObjectMeta:{2935k8s001    9931a1da-e618-4aa4-9401-7246e6c3b67b 4792 0 2022-05-03 07:47:00 +0000 UTC <nil> <nil> map[agentpool:windowspool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:windows failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:1 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:2935k8s001 kubernetes.io/os:windows kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 node.kubernetes.io/windows-build:10.0.17763 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:1] map[csi.volume.kubernetes.io/nodeid:{"smb.csi.k8s.io":"2935k8s001"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kube-controller-manager Update v1 2022-05-03 07:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet.exe Update v1 2022-05-03 07:47:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:node.kubernetes.io/windows-build":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:47:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kubelet.exe Update v1 2022-05-03 08:08:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/2935k8s001,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{8589463552 0} {<nil>} 8388148Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{136912564224 0} {<nil>} 133703676Ki BinarySI},memory: {{6441979904 0} {<nil>} 6290996Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:29:06 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:29:06 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:29:06 +0000 UTC,LastTransitionTime:2022-05-03 07:47:00 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:29:06 +0000 UTC,LastTransitionTime:2022-05-03 07:47:10 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:2935k8s001,},NodeAddress{Type:InternalIP,Address:10.240.0.65,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2935k8s001,SystemUUID:CF6606F6-7E65-4189-B311-9E8E8C38EE4D,BootID:,KernelVersion:10.0.17763.1879,OSImage:Windows Server 2019 Datacenter,ContainerRuntimeVersion:docker://20.10.9,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:windows,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/windows/servercore@sha256:1bb6543a7fe87bb95d9adfa4c53ea3469c1c846ad63baf32729078062e8c1127 mcr.microsoft.com/windows/servercore:ltsc2019],SizeBytes:5265856881,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:5ecb6e4f2ad6e12e153c396c68c2e412fc802514444d7ec7808806a50a340fe7 k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:287591069,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:4fd21f36075b44d1a423dfb262ad79202ce54e95f5cbc4622a6c1c38ab287ad6 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.0],SizeBytes:275364068,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:273371876,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:92dd20279c1e36282bd52e3db2679297921d517ffe4e98025c0370ce25a03fac mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v1.2.1-alpha.1-windows-1809-amd64],SizeBytes:265664693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:d07f92ef2517c99cd011391ab1182c995cdd04ec01e47324abf07ea502476409 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.0.1-alpha.1-windows-1809-amd64],SizeBytes:264960693,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e075a40446ece06b77075d247bbf027c039c26b956418b1c5b605330be1217f9 mcr.microsoft.com/oss/kubernetes/pause:1.4.1],SizeBytes:256028579,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 kubletwin/pause:latest mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:254580394,},ContainerImage{Names:[mcr.microsoft.com/windows/nanoserver@sha256:8746dc8b6726d53ba7176332580712c2c5622290b2b44345ded8e302356203ac mcr.microsoft.com/windows/nanoserver:1809],SizeBytes:251866052,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:31:24.424: INFO: 
Logging kubelet events for node 2935k8s001
May  3 08:31:24.458: INFO: 
Logging pods the kubelet thinks is on node 2935k8s001
May  3 08:31:24.540: INFO: csi-smb-node-win-8l4q4 started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:31:24.540: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:31:24.540: INFO: 	Container node-driver-registrar ready: true, restart count 0
May  3 08:31:24.540: INFO: 	Container smb ready: true, restart count 0
May  3 08:31:24.745: INFO: 
Latency metrics for node 2935k8s001
May  3 08:31:24.745: INFO: 
Logging node info for node k8s-master-29351631-0
May  3 08:31:24.780: INFO: Node Info: &Node{ObjectMeta:{k8s-master-29351631-0    4e179e14-aca2-4f8b-b6a0-9e9da86d6233 4777 0 2022-05-03 07:44:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:eastus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-rylebj4k kubernetes.azure.com/role:master kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-master-29351631-0 kubernetes.io/os:linux kubernetes.io/role:master node-role.kubernetes.io/master: node.kubernetes.io/exclude-disruption:true node.kubernetes.io/exclude-from-external-load-balancers:true node.kubernetes.io/instance-type:Standard_D2_v3 topology.kubernetes.io/region:eastus topology.kubernetes.io/zone:0] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2022-05-03 07:44:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/exclude-disruption":{},"f:node.kubernetes.io/exclude-from-external-load-balancers":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubectl-label Update v1 2022-05-03 07:44:56 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/master":{}}}} } {kube-controller-manager Update v1 2022-05-03 07:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {kubelet Update v1 2022-05-03 07:45:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-rylebj4k/providers/Microsoft.Compute/virtualMachines/k8s-master-29351631-0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344731648 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{attachable-volumes-azure-disk: {{4 0} {<nil>} 4 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558299648 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-05-03 08:28:57 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-05-03 08:28:57 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-05-03 08:28:57 +0000 UTC,LastTransitionTime:2022-05-03 07:44:48 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-05-03 08:28:57 +0000 UTC,LastTransitionTime:2022-05-03 07:45:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:Hostname,Address:k8s-master-29351631-0,},NodeAddress{Type:InternalIP,Address:10.255.255.5,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:05c3b26f052f42068b637e8504dfd0ed,SystemUUID:60ccb959-1914-ba42-a701-7a4085e0a4b4,BootID:e4b8bab9-5341-4ef3-b060-29ab390539e4,KernelVersion:5.4.0-1074-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.11+azure-3,KubeletVersion:v1.22.8,KubeProxyVersion:v1.22.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[k8sprow.azurecr.io/smb-csi@sha256:3c8fd9a8710f05be99405947ada32cd9e4a6657349e32873172da61dd2ba56ba k8sprow.azurecr.io/smb-csi:e2e-29246e6ee06db7440d2495ffbb71d22255e2ecb2],SizeBytes:190805704,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver@sha256:808a6792d8bbfbe22951ac274b3931c2b4484e0e6e09cdf93c8c887b25257c76 mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.8],SizeBytes:128319182,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager@sha256:298409ea9430b099d031ee92f9a128ab85d9531977ac0a30f651eb769f66dd7d mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.8],SizeBytes:122007567,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:29c53a2a5c90a860e9ab8dcff1f4bc93b70781958de71611a711fcaaea3b09ef mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.8],SizeBytes:103650551,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:122bfb8c1edabb3c0edd63f06523e6940d958d19b3957dc7b1d6f81e9f1f6119 registry.k8s.io/sig-storage/csi-provisioner:v3.1.0],SizeBytes:57729376,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[andyzhangx/samba@sha256:1c268eae7ec1eb9fbbbd4f0aab3ee41c7a4c4ecb0313c498cb11cfdcc762ad3e andyzhangx/samba:win-fix],SizeBytes:54262056,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler@sha256:cf4a41eb7c3f844556059845a9d486e8b1e407a61285d70093fa1056d4bbb91b mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.8],SizeBytes:52670775,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:0ad67f9919522a07318034641ae09bf2079b417e9944c65914410594ce645468 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.4],SizeBytes:51478397,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:31ec4f7daccd3e7a8504e12657d7830651ecacbe4a487daca1b1b7695a64b070 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.1],SizeBytes:51249021,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:011712ed90fb8efcf27928b0a47ed04b98baebb31cb1b2d8ab676977ec18eedc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.4],SizeBytes:50868093,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:0f9a8fbaed65192ed7dd795be4f9c1dc48ebdef0a241fb62d456f4bed40d9875 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.1],SizeBytes:50679677,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:7c907ff70b90a0bdf8fae63bd744018469dd9839cde1fd0515b93e0bbd14b34e mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.8],SizeBytes:48963453,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3987d7a8c6922ce1952ee19c5cb6ea75aac7b7c1b07aa79277ad038c69fb7a31 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.8],SizeBytes:48349053,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:8073113a20882642a980b338635cdc5945e5673a18aef192090e6fde2b89a75c mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.6.0],SizeBytes:45909032,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:6a32329628bdea3c6d75e98aad6155b65d2e2b98ca616eb33f9ac562912804c6 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.6.0],SizeBytes:45229096,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:ef6c4ba564b4d11d270f7d1563c50fbeb30ccc3b94146e5059228c49f95875f5 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.7.11],SizeBytes:44916605,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:dbcc384758ba5ca6d249596d471292ed3785e31cdb854d48b84d70794b669b4c mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.7.11],SizeBytes:43679613,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:406f59599991916d2942d8d02f076d957ed71b541ee19f09fc01723a6e6f5932 registry.k8s.io/sig-storage/livenessprobe:v2.6.0],SizeBytes:17605286,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
May  3 08:31:24.780: INFO: 
Logging kubelet events for node k8s-master-29351631-0
May  3 08:31:24.814: INFO: 
Logging pods the kubelet thinks is on node k8s-master-29351631-0
May  3 08:31:24.886: INFO: metrics-server-7bf8c745cc-2vkbr started at 2022-05-03 07:46:00 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container metrics-server ready: true, restart count 0
May  3 08:31:24.886: INFO: smb-server-589b6679cb-hnz5f started at 2022-05-03 07:48:31 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container smb-server ready: true, restart count 0
May  3 08:31:24.886: INFO: csi-smb-controller-856896f755-fnl5f started at 2022-05-03 08:07:44 +0000 UTC (0+3 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container csi-provisioner ready: true, restart count 0
May  3 08:31:24.886: INFO: 	Container liveness-probe ready: true, restart count 0
May  3 08:31:24.886: INFO: 	Container smb ready: true, restart count 0
May  3 08:31:24.886: INFO: kube-scheduler-k8s-master-29351631-0 started at 2022-05-03 07:44:14 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container kube-scheduler ready: true, restart count 0
May  3 08:31:24.886: INFO: kube-apiserver-k8s-master-29351631-0 started at 2022-05-03 07:44:15 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container kube-apiserver ready: true, restart count 0
May  3 08:31:24.886: INFO: coredns-autoscaler-66d6fbf7c7-2t68r started at 2022-05-03 07:45:46 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container autoscaler ready: true, restart count 0
May  3 08:31:24.886: INFO: kube-proxy-hqv66 started at 2022-05-03 07:45:51 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container kube-proxy ready: true, restart count 0
May  3 08:31:24.886: INFO: kube-addon-manager-k8s-master-29351631-0 started at 2022-05-03 07:45:39 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container kube-addon-manager ready: true, restart count 0
May  3 08:31:24.886: INFO: kube-controller-manager-k8s-master-29351631-0 started at 2022-05-03 07:44:15 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container kube-controller-manager ready: true, restart count 0
May  3 08:31:24.886: INFO: coredns-745f545b98-rnmxq started at 2022-05-03 07:45:46 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container coredns ready: true, restart count 0
May  3 08:31:24.886: INFO: azure-ip-masq-agent-zfzpr started at 2022-05-03 07:45:51 +0000 UTC (0+1 container statuses recorded)
May  3 08:31:24.886: INFO: 	Container azure-ip-masq-agent ready: true, restart count 0
May  3 08:31:25.021: INFO: 
Latency metrics for node k8s-master-29351631-0
May  3 08:31:25.022: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
�[1mSTEP�[0m: Destroying namespace "smb-694" for this suite.

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

kubetest Test 53m5s

error during make e2e-test: exit status 2
				from junit_runner.xml

Filter through log files | View test history on testgrid

PR	andyzhangx: chore: replace mcr images with gcr.io images
Result	FAILURE
Tests	4 failed / 19 succeeded
Started	2022-05-03 07:41
Elapsed	1h7m
Revision	4e92c7394033a791167b61fd629debf9e1cacc96
Refs	475
job-version	v1.25.0-alpha.0.15+e7a2ce75e5df96
kubetest-version
revision	v1.25.0-alpha.0.15+e7a2ce75e5df96

kubernetes-csi/csi-driver-smb PR #475 pull-csi-driver-smb-e2e-windows #1521393877724958720

Test Failures

SMB CSI Driver End-to-End Tests Dynamic Provisioning [createSubDir] should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [smb.csi.k8s.io] [Windows] 7m6s

SMB CSI Driver End-to-End Tests Dynamic Provisioning should create a volume on demand with mount options [smb.csi.k8s.io] [Windows] 15m46s

SMB CSI Driver End-to-End Tests Dynamic Provisioning should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [smb.csi.k8s.io] [Windows] 6m40s

kubetest Test 53m5s

Passed

Skipped