This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2021-06-14 16:06
Elapsed32m0s
Revisionmaster

No Test Failures!


Error lines from build-log.txt

... skipping 501 lines ...
I0614 16:13:17.865287   12785 up.go:43] Cleaning up any leaked resources from previous cluster
I0614 16:13:17.865307   12785 dumplogs.go:38] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops toolbox dump --name e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I0614 16:13:17.898582   12791 featureflag.go:167] FeatureFlag "SpecOverrideFlag"=true
I0614 16:13:17.898734   12791 featureflag.go:167] FeatureFlag "AlphaAllowGCE"=true

Cluster.kops.k8s.io "e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io" not found
W0614 16:13:18.447557   12785 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I0614 16:13:18.447602   12785 down.go:48] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops delete cluster --name e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --yes
I0614 16:13:18.463874   12801 featureflag.go:167] FeatureFlag "SpecOverrideFlag"=true
I0614 16:13:18.463960   12801 featureflag.go:167] FeatureFlag "AlphaAllowGCE"=true

error reading cluster configuration: Cluster.kops.k8s.io "e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io" not found
I0614 16:13:18.994727   12785 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2021/06/14 16:13:19 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I0614 16:13:19.025946   12785 http.go:37] curl https://ip.jsb.workers.dev
I0614 16:13:19.296064   12785 up.go:144] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops create cluster --name e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --cloud aws --kubernetes-version 1.21.0 --ssh-public-key /etc/aws-ssh/aws-ssh-public --override cluster.spec.nodePortAccess=0.0.0.0/0 --yes --networking calico --override=cluster.spec.cloudConfig.awsEBSCSIDriver.enabled=true --override=cluster.spec.snapshotController.enabled=true --override=cluster.spec.certManager.enabled=true --admin-access 34.71.50.30/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones ap-northeast-1a --master-size c5.large
I0614 16:13:19.309951   12815 featureflag.go:167] FeatureFlag "SpecOverrideFlag"=true
I0614 16:13:19.310044   12815 featureflag.go:167] FeatureFlag "AlphaAllowGCE"=true
I0614 16:13:19.366294   12815 create_cluster.go:732] Using SSH public key: /etc/aws-ssh/aws-ssh-public
I0614 16:13:19.831995   12815 new_cluster.go:1054]  Cloud Provider ID = aws
... skipping 40 lines ...

I0614 16:13:50.309937   12785 up.go:181] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops validate cluster --name e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --count 10 --wait 15m0s
I0614 16:13:50.325664   12835 featureflag.go:167] FeatureFlag "SpecOverrideFlag"=true
I0614 16:13:50.326428   12835 featureflag.go:167] FeatureFlag "AlphaAllowGCE"=true
Validating cluster e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io

W0614 16:13:51.832478   12835 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:14:01.877332   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
W0614 16:14:11.907402   12835 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:14:21.951723   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:14:31.987551   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:14:42.050717   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:14:52.102426   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:15:02.147403   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:15:12.179757   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:15:22.211918   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:15:32.244520   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:15:42.273185   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:15:52.309120   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:16:02.373162   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:16:12.404196   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:16:22.434392   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:16:32.473611   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:16:42.520323   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:16:52.559243   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:17:02.610852   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:17:12.654393   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:17:22.688368   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:17:32.719489   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

NODE STATUS
NAME	ROLE	READY

VALIDATION ERRORS
KIND	NAME		MESSAGE
dns	apiserver	Validation Failed

The dns-controller Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address.  The API DNS IP address is the placeholder address that kops creates: 203.0.113.123.  Please wait about 5-10 minutes for a master to start, dns-controller to launch, and DNS to propagate.  The protokube container and dns-controller deployment logs may contain more diagnostic information.  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.

Validation Failed
W0614 16:17:42.767014   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 15 lines ...
Pod	kube-system/coredns-autoscaler-6f594f4c58-xvgxz		system-cluster-critical pod "coredns-autoscaler-6f594f4c58-xvgxz" is pending
Pod	kube-system/coredns-f45c4bf76-rzzzf			system-cluster-critical pod "coredns-f45c4bf76-rzzzf" is pending
Pod	kube-system/ebs-csi-controller-7ffff4ddf5-j2k6c		system-cluster-critical pod "ebs-csi-controller-7ffff4ddf5-j2k6c" is pending
Pod	kube-system/ebs-csi-controller-7ffff4ddf5-ljdnk		system-cluster-critical pod "ebs-csi-controller-7ffff4ddf5-ljdnk" is pending
Pod	kube-system/ebs-csi-node-lw2n4				system-node-critical pod "ebs-csi-node-lw2n4" is pending

Validation Failed
W0614 16:17:55.994682   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 18 lines ...
Pod	kube-system/coredns-f45c4bf76-rzzzf					system-cluster-critical pod "coredns-f45c4bf76-rzzzf" is pending
Pod	kube-system/ebs-csi-controller-7ffff4ddf5-j2k6c				system-cluster-critical pod "ebs-csi-controller-7ffff4ddf5-j2k6c" is pending
Pod	kube-system/ebs-csi-controller-7ffff4ddf5-ljdnk				system-cluster-critical pod "ebs-csi-controller-7ffff4ddf5-ljdnk" is pending
Pod	kube-system/ebs-csi-node-nwl9v						system-node-critical pod "ebs-csi-node-nwl9v" is pending
Pod	kube-system/kube-proxy-ip-172-20-51-223.ap-northeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-51-223.ap-northeast-1.compute.internal" is pending

Validation Failed
W0614 16:18:08.334406   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 23 lines ...
Pod	kube-system/ebs-csi-node-8bvzp						system-node-critical pod "ebs-csi-node-8bvzp" is pending
Pod	kube-system/ebs-csi-node-hbbz5						system-node-critical pod "ebs-csi-node-hbbz5" is pending
Pod	kube-system/ebs-csi-node-nwl9v						system-node-critical pod "ebs-csi-node-nwl9v" is pending
Pod	kube-system/kube-proxy-ip-172-20-37-65.ap-northeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-37-65.ap-northeast-1.compute.internal" is pending
Pod	kube-system/kube-proxy-ip-172-20-61-58.ap-northeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-61-58.ap-northeast-1.compute.internal" is pending

Validation Failed
W0614 16:18:20.634912   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 22 lines ...
Pod	kube-system/ebs-csi-node-8bvzp						system-node-critical pod "ebs-csi-node-8bvzp" is pending
Pod	kube-system/ebs-csi-node-8gm7m						system-node-critical pod "ebs-csi-node-8gm7m" is pending
Pod	kube-system/ebs-csi-node-hbbz5						system-node-critical pod "ebs-csi-node-hbbz5" is pending
Pod	kube-system/ebs-csi-node-nwl9v						system-node-critical pod "ebs-csi-node-nwl9v" is pending
Pod	kube-system/kube-proxy-ip-172-20-56-37.ap-northeast-1.compute.internal	system-node-critical pod "kube-proxy-ip-172-20-56-37.ap-northeast-1.compute.internal" is pending

Validation Failed
W0614 16:18:33.105714   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 16 lines ...
Pod	kube-system/cert-manager-webhook-7bbf67968-rbc94	system-cluster-critical pod "cert-manager-webhook-7bbf67968-rbc94" is pending
Pod	kube-system/coredns-autoscaler-6f594f4c58-xvgxz		system-cluster-critical pod "coredns-autoscaler-6f594f4c58-xvgxz" is pending
Pod	kube-system/coredns-f45c4bf76-rzzzf			system-cluster-critical pod "coredns-f45c4bf76-rzzzf" is pending
Pod	kube-system/ebs-csi-node-8bvzp				system-node-critical pod "ebs-csi-node-8bvzp" is pending
Pod	kube-system/ebs-csi-node-8gm7m				system-node-critical pod "ebs-csi-node-8gm7m" is pending

Validation Failed
W0614 16:18:45.413547   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 13 lines ...
Pod	kube-system/calico-node-qf7pt				system-node-critical pod "calico-node-qf7pt" is not ready (calico-node)
Pod	kube-system/cert-manager-cainjector-74d69756d5-csqwc	system-cluster-critical pod "cert-manager-cainjector-74d69756d5-csqwc" is pending
Pod	kube-system/cert-manager-webhook-7bbf67968-rbc94	system-cluster-critical pod "cert-manager-webhook-7bbf67968-rbc94" is pending
Pod	kube-system/coredns-autoscaler-6f594f4c58-xvgxz		system-cluster-critical pod "coredns-autoscaler-6f594f4c58-xvgxz" is pending
Pod	kube-system/coredns-f45c4bf76-rzzzf			system-cluster-critical pod "coredns-f45c4bf76-rzzzf" is pending

Validation Failed
W0614 16:18:57.645122   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 9 lines ...
KIND	NAME							MESSAGE
Pod	kube-system/calico-node-crggb				system-node-critical pod "calico-node-crggb" is not ready (calico-node)
Pod	kube-system/cert-manager-webhook-7bbf67968-rbc94	system-cluster-critical pod "cert-manager-webhook-7bbf67968-rbc94" is pending
Pod	kube-system/coredns-autoscaler-6f594f4c58-xvgxz		system-cluster-critical pod "coredns-autoscaler-6f594f4c58-xvgxz" is pending
Pod	kube-system/coredns-f45c4bf76-ks592			system-cluster-critical pod "coredns-f45c4bf76-ks592" is pending

Validation Failed
W0614 16:19:10.172303   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 7 lines ...

VALIDATION ERRORS
KIND	NAME							MESSAGE
Pod	kube-system/calico-node-crggb				system-node-critical pod "calico-node-crggb" is not ready (calico-node)
Pod	kube-system/cert-manager-webhook-7bbf67968-rbc94	system-cluster-critical pod "cert-manager-webhook-7bbf67968-rbc94" is pending

Validation Failed
W0614 16:19:22.464651   12835 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-ap-northeast-1a	Master	c5.large	1	1	ap-northeast-1a
nodes-ap-northeast-1a	Node	t3.medium	4	4	ap-northeast-1a

... skipping 294 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Pre-provisioned PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256

      Driver supports dynamic provisioning, skipping PreprovisionedPV pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 189 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Inline-volume (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256

    Driver "ebs.csi.aws.com" does not support volume type "InlineVolume" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 122 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Pre-provisioned PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256

    Driver "ebs.csi.aws.com" does not support volume type "PreprovisionedPV" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 575 lines ...
Jun 14 16:22:55.052: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
STEP: creating a test aws volume
Jun 14 16:22:56.015: INFO: Successfully created a new PD: "aws://ap-northeast-1a/vol-0bb1060df3c911aaa".
Jun 14 16:22:56.015: INFO: Creating resource for inline volume
STEP: Creating pod exec-volume-test-inlinevolume-wd5k
STEP: Creating a pod to test exec-volume-test
Jun 14 16:22:56.151: INFO: Waiting up to 5m0s for pod "exec-volume-test-inlinevolume-wd5k" in namespace "volume-2269" to be "Succeeded or Failed"
Jun 14 16:22:56.284: INFO: Pod "exec-volume-test-inlinevolume-wd5k": Phase="Pending", Reason="", readiness=false. Elapsed: 133.651499ms
Jun 14 16:22:58.419: INFO: Pod "exec-volume-test-inlinevolume-wd5k": Phase="Pending", Reason="", readiness=false. Elapsed: 2.267955676s
Jun 14 16:23:00.552: INFO: Pod "exec-volume-test-inlinevolume-wd5k": Phase="Pending", Reason="", readiness=false. Elapsed: 4.401235485s
Jun 14 16:23:02.686: INFO: Pod "exec-volume-test-inlinevolume-wd5k": Phase="Pending", Reason="", readiness=false. Elapsed: 6.534849882s
Jun 14 16:23:04.819: INFO: Pod "exec-volume-test-inlinevolume-wd5k": Phase="Pending", Reason="", readiness=false. Elapsed: 8.667918384s
Jun 14 16:23:06.954: INFO: Pod "exec-volume-test-inlinevolume-wd5k": Phase="Pending", Reason="", readiness=false. Elapsed: 10.802944108s
Jun 14 16:23:09.087: INFO: Pod "exec-volume-test-inlinevolume-wd5k": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.936184284s
STEP: Saw pod success
Jun 14 16:23:09.087: INFO: Pod "exec-volume-test-inlinevolume-wd5k" satisfied condition "Succeeded or Failed"
Jun 14 16:23:09.220: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod exec-volume-test-inlinevolume-wd5k container exec-container-inlinevolume-wd5k: <nil>
STEP: delete the pod
Jun 14 16:23:09.508: INFO: Waiting for pod exec-volume-test-inlinevolume-wd5k to disappear
Jun 14 16:23:09.642: INFO: Pod exec-volume-test-inlinevolume-wd5k no longer exists
STEP: Deleting pod exec-volume-test-inlinevolume-wd5k
Jun 14 16:23:09.642: INFO: Deleting pod "exec-volume-test-inlinevolume-wd5k" in namespace "volume-2269"
Jun 14 16:23:10.071: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0bb1060df3c911aaa", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0bb1060df3c911aaa is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 768daedc-0d00-41f9-a5ed-910367c63e4d
Jun 14 16:23:15.861: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0bb1060df3c911aaa", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0bb1060df3c911aaa is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 92efae91-66fd-459d-b9b4-67bc0ecf8e2a
Jun 14 16:23:21.578: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0bb1060df3c911aaa", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0bb1060df3c911aaa is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 928af71e-432c-40d6-846a-ca9b5d3a1d82
Jun 14 16:23:27.329: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-0bb1060df3c911aaa".
[AfterEach] [Testpattern: Inline-volume (xfs)][Slow] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:23:27.329: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-2269" for this suite.
... skipping 78 lines ...
Jun 14 16:22:54.590: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass volume-8008-e2e-sc29b2s
STEP: creating a claim
Jun 14 16:22:54.720: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod exec-volume-test-dynamicpv-87sc
STEP: Creating a pod to test exec-volume-test
Jun 14 16:22:55.113: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-87sc" in namespace "volume-8008" to be "Succeeded or Failed"
Jun 14 16:22:55.245: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 131.871473ms
Jun 14 16:22:57.375: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 2.262677053s
Jun 14 16:22:59.507: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 4.394204565s
Jun 14 16:23:01.636: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 6.523047738s
Jun 14 16:23:03.767: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 8.653972138s
Jun 14 16:23:05.896: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 10.782850858s
... skipping 3 lines ...
Jun 14 16:23:14.420: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 19.306916879s
Jun 14 16:23:16.550: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 21.437352695s
Jun 14 16:23:18.679: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 23.566169739s
Jun 14 16:23:20.808: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Pending", Reason="", readiness=false. Elapsed: 25.695171854s
Jun 14 16:23:22.937: INFO: Pod "exec-volume-test-dynamicpv-87sc": Phase="Succeeded", Reason="", readiness=false. Elapsed: 27.824062656s
STEP: Saw pod success
Jun 14 16:23:22.937: INFO: Pod "exec-volume-test-dynamicpv-87sc" satisfied condition "Succeeded or Failed"
Jun 14 16:23:23.066: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod exec-volume-test-dynamicpv-87sc container exec-container-dynamicpv-87sc: <nil>
STEP: delete the pod
Jun 14 16:23:23.335: INFO: Waiting for pod exec-volume-test-dynamicpv-87sc to disappear
Jun 14 16:23:23.464: INFO: Pod exec-volume-test-dynamicpv-87sc no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-87sc
Jun 14 16:23:23.464: INFO: Deleting pod "exec-volume-test-dynamicpv-87sc" in namespace "volume-8008"
... skipping 28 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Pre-provisioned PV (block volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to use a volume in a pod with mismatched mode [Slow] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296

    Driver "ebs.csi.aws.com" does not support volume type "PreprovisionedPV" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 61 lines ...
STEP: Creating a kubernetes client
Jun 14 16:22:50.809: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
W0614 16:22:51.678916   26753 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Jun 14 16:22:51.679: INFO: No PodSecurityPolicies found; assuming PodSecurityPolicy is disabled.
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if subpath directory is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240
Jun 14 16:22:51.949: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:22:51.949: INFO: Creating resource for dynamic PV
Jun 14 16:22:51.949: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-5929zrqkh
STEP: creating a claim
Jun 14 16:22:52.085: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-plxr
STEP: Checking for subpath error in container status
Jun 14 16:23:20.786: INFO: Deleting pod "pod-subpath-test-dynamicpv-plxr" in namespace "provisioning-5929"
Jun 14 16:23:20.925: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-plxr" to be fully deleted
STEP: Deleting pod
Jun 14 16:23:31.197: INFO: Deleting pod "pod-subpath-test-dynamicpv-plxr" in namespace "provisioning-5929"
STEP: Deleting pvc
Jun 14 16:23:31.604: INFO: Deleting PersistentVolumeClaim "awsfwldj"
... skipping 12 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath directory is outside the volume [Slow][LinuxOnly]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:22:54.246: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if subpath directory is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240
Jun 14 16:22:54.935: INFO: Creating resource for dynamic PV
Jun 14 16:22:54.935: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-2353-e2e-scqxc7t
STEP: creating a claim
Jun 14 16:22:55.066: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-pnnh
STEP: Checking for subpath error in container status
Jun 14 16:23:23.712: INFO: Deleting pod "pod-subpath-test-dynamicpv-pnnh" in namespace "provisioning-2353"
Jun 14 16:23:23.840: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-pnnh" to be fully deleted
STEP: Deleting pod
Jun 14 16:23:32.093: INFO: Deleting pod "pod-subpath-test-dynamicpv-pnnh" in namespace "provisioning-2353"
STEP: Deleting pvc
Jun 14 16:23:32.471: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com4crp4"
... skipping 11 lines ...

• [SLOW TEST:54.245 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath directory is outside the volume [Slow][LinuxOnly]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:22:51.027: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
W0614 16:22:52.912275   26828 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
Jun 14 16:22:52.912: INFO: No PodSecurityPolicies found; assuming PodSecurityPolicy is disabled.
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if subpath file is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256
Jun 14 16:22:53.168: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:22:53.168: INFO: Creating resource for dynamic PV
Jun 14 16:22:53.168: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-5720jhxlv
STEP: creating a claim
Jun 14 16:22:53.297: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-hcxl
STEP: Checking for subpath error in container status
Jun 14 16:23:21.945: INFO: Deleting pod "pod-subpath-test-dynamicpv-hcxl" in namespace "provisioning-5720"
Jun 14 16:23:22.077: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-hcxl" to be fully deleted
STEP: Deleting pod
Jun 14 16:23:32.331: INFO: Deleting pod "pod-subpath-test-dynamicpv-hcxl" in namespace "provisioning-5720"
STEP: Deleting pvc
Jun 14 16:23:32.711: INFO: Deleting PersistentVolumeClaim "aws2qgcw"
... skipping 13 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath file is outside the volume [Slow][LinuxOnly]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256
------------------------------
[BeforeEach] [sig-storage] Zone Support [Feature:vsphere]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:23:48.861: INFO: >>> kubeConfig: /root/.kube/config
... skipping 77 lines ...
Jun 14 16:22:51.852: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass volume-7999z6dhc
STEP: creating a claim
Jun 14 16:22:51.980: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod exec-volume-test-dynamicpv-l7w7
STEP: Creating a pod to test exec-volume-test
Jun 14 16:22:52.377: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-l7w7" in namespace "volume-7999" to be "Succeeded or Failed"
Jun 14 16:22:52.513: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 135.959065ms
Jun 14 16:22:54.643: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.265585875s
Jun 14 16:22:56.772: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.394770231s
Jun 14 16:22:58.900: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.523131634s
Jun 14 16:23:01.062: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.68430132s
Jun 14 16:23:03.190: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.812926287s
Jun 14 16:23:05.320: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 12.942230273s
Jun 14 16:23:07.449: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 15.071920355s
Jun 14 16:23:09.578: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 17.201162653s
Jun 14 16:23:11.708: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 19.330577818s
Jun 14 16:23:13.837: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Pending", Reason="", readiness=false. Elapsed: 21.460035496s
Jun 14 16:23:15.969: INFO: Pod "exec-volume-test-dynamicpv-l7w7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.59183799s
STEP: Saw pod success
Jun 14 16:23:15.969: INFO: Pod "exec-volume-test-dynamicpv-l7w7" satisfied condition "Succeeded or Failed"
Jun 14 16:23:16.097: INFO: Trying to get logs from node ip-172-20-51-223.ap-northeast-1.compute.internal pod exec-volume-test-dynamicpv-l7w7 container exec-container-dynamicpv-l7w7: <nil>
STEP: delete the pod
Jun 14 16:23:16.570: INFO: Waiting for pod exec-volume-test-dynamicpv-l7w7 to disappear
Jun 14 16:23:16.697: INFO: Pod exec-volume-test-dynamicpv-l7w7 no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-l7w7
Jun 14 16:23:16.697: INFO: Deleting pod "exec-volume-test-dynamicpv-l7w7" in namespace "volume-7999"
... skipping 658 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278

      Distro debian doesn't support ntfs -- skipping

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:127
------------------------------
... skipping 10 lines ...
Jun 14 16:23:28.288: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
STEP: creating a test aws volume
Jun 14 16:23:28.717: INFO: Successfully created a new PD: "aws://ap-northeast-1a/vol-066728bb623d146b7".
Jun 14 16:23:28.717: INFO: Creating resource for inline volume
STEP: Creating pod exec-volume-test-inlinevolume-wqvm
STEP: Creating a pod to test exec-volume-test
Jun 14 16:23:28.856: INFO: Waiting up to 5m0s for pod "exec-volume-test-inlinevolume-wqvm" in namespace "volume-4651" to be "Succeeded or Failed"
Jun 14 16:23:28.992: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 136.45235ms
Jun 14 16:23:31.128: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.272805392s
Jun 14 16:23:33.268: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.41249771s
Jun 14 16:23:35.405: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.549508587s
Jun 14 16:23:37.541: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.685720049s
Jun 14 16:23:39.678: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 10.822473173s
... skipping 7 lines ...
Jun 14 16:23:56.791: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 27.935134166s
Jun 14 16:23:58.928: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 30.072514504s
Jun 14 16:24:01.065: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 32.209690825s
Jun 14 16:24:03.203: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Pending", Reason="", readiness=false. Elapsed: 34.347368853s
Jun 14 16:24:05.341: INFO: Pod "exec-volume-test-inlinevolume-wqvm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 36.485403788s
STEP: Saw pod success
Jun 14 16:24:05.341: INFO: Pod "exec-volume-test-inlinevolume-wqvm" satisfied condition "Succeeded or Failed"
Jun 14 16:24:05.480: INFO: Trying to get logs from node ip-172-20-56-37.ap-northeast-1.compute.internal pod exec-volume-test-inlinevolume-wqvm container exec-container-inlinevolume-wqvm: <nil>
STEP: delete the pod
Jun 14 16:24:05.764: INFO: Waiting for pod exec-volume-test-inlinevolume-wqvm to disappear
Jun 14 16:24:05.901: INFO: Pod exec-volume-test-inlinevolume-wqvm no longer exists
STEP: Deleting pod exec-volume-test-inlinevolume-wqvm
Jun 14 16:24:05.901: INFO: Deleting pod "exec-volume-test-inlinevolume-wqvm" in namespace "volume-4651"
Jun 14 16:24:06.322: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-066728bb623d146b7", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-066728bb623d146b7 is currently attached to i-02f6d88db741a2194
	status code: 400, request id: f2a03e88-a725-499a-a38f-6024a908f0af
Jun 14 16:24:12.104: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-066728bb623d146b7", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-066728bb623d146b7 is currently attached to i-02f6d88db741a2194
	status code: 400, request id: 4756b690-71af-4fe6-8e64-b8e2f4cc6c09
Jun 14 16:24:17.998: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-066728bb623d146b7".
[AfterEach] [Testpattern: Inline-volume (ext4)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:24:17.998: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-4651" for this suite.
... skipping 39 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Pre-provisioned PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278

    Driver "ebs.csi.aws.com" does not support volume type "PreprovisionedPV" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 139 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Inline-volume (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240

      Driver supports dynamic provisioning, skipping InlineVolume pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 168 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (immediate binding)] topology
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver "ebs.csi.aws.com" does not support topology - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:92
------------------------------
... skipping 75 lines ...
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:23:33.033: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267
Jun 14 16:23:33.659: INFO: Creating resource for dynamic PV
Jun 14 16:23:33.659: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-46-e2e-sc28czr
STEP: creating a claim
Jun 14 16:23:33.786: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-7xch
STEP: Checking for subpath error in container status
Jun 14 16:23:58.418: INFO: Deleting pod "pod-subpath-test-dynamicpv-7xch" in namespace "provisioning-46"
Jun 14 16:23:58.545: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-7xch" to be fully deleted
STEP: Deleting pod
Jun 14 16:24:06.822: INFO: Deleting pod "pod-subpath-test-dynamicpv-7xch" in namespace "provisioning-46"
STEP: Deleting pvc
Jun 14 16:24:07.197: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comtcdh9"
... skipping 11 lines ...

• [SLOW TEST:50.175 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Jun 14 16:24:23.209: INFO: Driver ebs.csi.aws.com doesn't support ntfs -- skipping
[AfterEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning
... skipping 746 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240

      Distro debian doesn't support ntfs -- skipping

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:127
------------------------------
... skipping 67 lines ...
Jun 14 16:24:12.301: INFO: Pod aws-client still exists
Jun 14 16:24:14.169: INFO: Waiting for pod aws-client to disappear
Jun 14 16:24:14.299: INFO: Pod aws-client still exists
Jun 14 16:24:16.168: INFO: Waiting for pod aws-client to disappear
Jun 14 16:24:16.298: INFO: Pod aws-client no longer exists
STEP: cleaning the environment after aws
Jun 14 16:24:17.108: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-03bb04f73c981542f", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-03bb04f73c981542f is currently attached to i-02f6d88db741a2194
	status code: 400, request id: 9290e544-47b4-4c04-9801-fbcfe28bcc23
Jun 14 16:24:22.875: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-03bb04f73c981542f", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-03bb04f73c981542f is currently attached to i-02f6d88db741a2194
	status code: 400, request id: efd254f1-e871-4071-ad84-8b83eef99176
Jun 14 16:24:28.609: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-03bb04f73c981542f".
[AfterEach] [Testpattern: Inline-volume (xfs)][Slow] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:24:28.609: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-7940" for this suite.
... skipping 391 lines ...
Jun 14 16:24:04.851: INFO: PersistentVolumeClaim pvc-gfwnh found but phase is Pending instead of Bound.
Jun 14 16:24:06.978: INFO: PersistentVolumeClaim pvc-gfwnh found and phase=Bound (15.019402107s)
Jun 14 16:24:06.978: INFO: Waiting up to 3m0s for PersistentVolume aws-s5lwn to have phase Bound
Jun 14 16:24:07.104: INFO: PersistentVolume aws-s5lwn found and phase=Bound (126.72341ms)
STEP: Creating pod exec-volume-test-preprovisionedpv-7wzl
STEP: Creating a pod to test exec-volume-test
Jun 14 16:24:07.488: INFO: Waiting up to 5m0s for pod "exec-volume-test-preprovisionedpv-7wzl" in namespace "volume-4234" to be "Succeeded or Failed"
Jun 14 16:24:07.615: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 126.988949ms
Jun 14 16:24:09.743: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.254528578s
Jun 14 16:24:11.871: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.382169685s
Jun 14 16:24:13.998: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.509163342s
Jun 14 16:24:16.124: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.635905688s
Jun 14 16:24:18.253: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.764614744s
Jun 14 16:24:20.381: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 12.892347848s
Jun 14 16:24:22.508: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 15.019128735s
Jun 14 16:24:24.635: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Pending", Reason="", readiness=false. Elapsed: 17.147005415s
Jun 14 16:24:26.765: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl": Phase="Succeeded", Reason="", readiness=false. Elapsed: 19.276969144s
STEP: Saw pod success
Jun 14 16:24:26.765: INFO: Pod "exec-volume-test-preprovisionedpv-7wzl" satisfied condition "Succeeded or Failed"
Jun 14 16:24:26.905: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod exec-volume-test-preprovisionedpv-7wzl container exec-container-preprovisionedpv-7wzl: <nil>
STEP: delete the pod
Jun 14 16:24:27.199: INFO: Waiting for pod exec-volume-test-preprovisionedpv-7wzl to disappear
Jun 14 16:24:27.325: INFO: Pod exec-volume-test-preprovisionedpv-7wzl no longer exists
STEP: Deleting pod exec-volume-test-preprovisionedpv-7wzl
Jun 14 16:24:27.326: INFO: Deleting pod "exec-volume-test-preprovisionedpv-7wzl" in namespace "volume-4234"
STEP: Deleting pv and pvc
Jun 14 16:24:27.452: INFO: Deleting PersistentVolumeClaim "pvc-gfwnh"
Jun 14 16:24:27.580: INFO: Deleting PersistentVolume "aws-s5lwn"
Jun 14 16:24:28.050: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-015e31b8095c35a30", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-015e31b8095c35a30 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 3c7e440f-51ab-4726-b4c6-68c95538dcec
Jun 14 16:24:33.796: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-015e31b8095c35a30", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-015e31b8095c35a30 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: edb97649-fe6a-4a2c-b607-55cd04fcf1f3
Jun 14 16:24:39.517: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-015e31b8095c35a30", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-015e31b8095c35a30 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 604ddd07-2391-4569-bf8d-e86ec7ec4c2b
Jun 14 16:24:45.353: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-015e31b8095c35a30", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-015e31b8095c35a30 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: dd30b852-a701-4f23-a5be-cd19671dbe41
Jun 14 16:24:51.043: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-015e31b8095c35a30".
[AfterEach] [Testpattern: Pre-provisioned PV (ext4)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:24:51.043: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-4234" for this suite.
... skipping 26 lines ...
Jun 14 16:22:53.112: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-7576-e2e-sc7t4j7      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-7576    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7576-e2e-sc7t4j7,VolumeMode:nil,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-7576    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7576-e2e-sc7t4j7,VolumeMode:nil,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}
STEP: Creating a StorageClass
STEP: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-7576    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7576-e2e-sc7t4j7,VolumeMode:nil,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}
STEP: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-7576-e2e-sc7t4j7    7ab20ba1-6ae5-4145-af00-d941ef385ca3 2463 0 2021-06-14 16:22:53 +0000 UTC <nil> <nil> map[] map[] [] []  [{e2e-kubernetes.test Update storage.k8s.io/v1 2021-06-14 16:22:53 +0000 UTC FieldsV1 {"f:mountOptions":{},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}}}]},Provisioner:ebs.csi.aws.com,Parameters:map[string]string{},ReclaimPolicy:*Delete,MountOptions:[dirsync],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-5bgd8 pvc- provisioning-7576  2ce4863f-5ff3-406a-becf-21a01327ce6b 2483 0 2021-06-14 16:22:53 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection]  [{e2e-kubernetes.test Update v1 2021-06-14 16:22:53 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}}}]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-7576-e2e-sc7t4j7,VolumeMode:*Filesystem,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}
STEP: Deleting pod pod-7e64416c-1b1f-4d39-836a-1e0b05390c89 in namespace provisioning-7576
STEP: checking the created volume is writable on node {Name: Selector:map[] Affinity:nil}
Jun 14 16:23:24.508: INFO: Waiting up to 15m0s for pod "pvc-volume-tester-writer-pk85k" in namespace "provisioning-7576" to be "Succeeded or Failed"
Jun 14 16:23:24.632: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 123.334397ms
Jun 14 16:23:26.761: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 2.252700545s
Jun 14 16:23:28.885: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 4.376069763s
Jun 14 16:23:31.009: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 6.500430703s
Jun 14 16:23:33.133: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 8.624536705s
Jun 14 16:23:35.257: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 10.748413169s
... skipping 22 lines ...
Jun 14 16:24:24.164: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 59.655981343s
Jun 14 16:24:26.288: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 1m1.779825965s
Jun 14 16:24:28.413: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.904705875s
Jun 14 16:24:30.537: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.02857184s
Jun 14 16:24:32.662: INFO: Pod "pvc-volume-tester-writer-pk85k": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m8.153633731s
STEP: Saw pod success
Jun 14 16:24:32.662: INFO: Pod "pvc-volume-tester-writer-pk85k" satisfied condition "Succeeded or Failed"
Jun 14 16:24:32.913: INFO: Pod pvc-volume-tester-writer-pk85k has the following logs: 
Jun 14 16:24:32.913: INFO: Deleting pod "pvc-volume-tester-writer-pk85k" in namespace "provisioning-7576"
Jun 14 16:24:33.043: INFO: Wait up to 5m0s for pod "pvc-volume-tester-writer-pk85k" to be fully deleted
STEP: checking the created volume has the correct mount options, is readable and retains data on the same node "ip-172-20-51-223.ap-northeast-1.compute.internal"
Jun 14 16:24:33.547: INFO: Waiting up to 15m0s for pod "pvc-volume-tester-reader-x67qt" in namespace "provisioning-7576" to be "Succeeded or Failed"
Jun 14 16:24:33.670: INFO: Pod "pvc-volume-tester-reader-x67qt": Phase="Pending", Reason="", readiness=false. Elapsed: 123.048847ms
Jun 14 16:24:35.793: INFO: Pod "pvc-volume-tester-reader-x67qt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.246141065s
Jun 14 16:24:37.917: INFO: Pod "pvc-volume-tester-reader-x67qt": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.370812925s
STEP: Saw pod success
Jun 14 16:24:37.918: INFO: Pod "pvc-volume-tester-reader-x67qt" satisfied condition "Succeeded or Failed"
Jun 14 16:24:38.167: INFO: Pod pvc-volume-tester-reader-x67qt has the following logs: hello world

Jun 14 16:24:38.167: INFO: Deleting pod "pvc-volume-tester-reader-x67qt" in namespace "provisioning-7576"
Jun 14 16:24:38.301: INFO: Wait up to 5m0s for pod "pvc-volume-tester-reader-x67qt" to be fully deleted
Jun 14 16:24:38.424: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-5bgd8] to have phase Bound
Jun 14 16:24:38.548: INFO: PersistentVolumeClaim pvc-5bgd8 found and phase=Bound (124.503822ms)
... skipping 164 lines ...
Jun 14 16:23:40.554: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-8199-e2e-sc9kh8d
STEP: creating a claim
Jun 14 16:23:40.684: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-j8w2
STEP: Creating a pod to test subpath
Jun 14 16:23:41.074: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-j8w2" in namespace "provisioning-8199" to be "Succeeded or Failed"
Jun 14 16:23:41.203: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 128.727589ms
Jun 14 16:23:43.332: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.257882466s
Jun 14 16:23:45.462: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.38783753s
Jun 14 16:23:47.591: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.516881405s
Jun 14 16:23:49.721: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.646332884s
Jun 14 16:23:51.851: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.777059003s
... skipping 10 lines ...
Jun 14 16:24:15.280: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 34.205673655s
Jun 14 16:24:17.409: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 36.334855152s
Jun 14 16:24:19.547: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 38.472562019s
Jun 14 16:24:21.676: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Pending", Reason="", readiness=false. Elapsed: 40.602021545s
Jun 14 16:24:23.807: INFO: Pod "pod-subpath-test-dynamicpv-j8w2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 42.732492669s
STEP: Saw pod success
Jun 14 16:24:23.807: INFO: Pod "pod-subpath-test-dynamicpv-j8w2" satisfied condition "Succeeded or Failed"
Jun 14 16:24:23.936: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-j8w2 container test-container-subpath-dynamicpv-j8w2: <nil>
STEP: delete the pod
Jun 14 16:24:24.205: INFO: Waiting for pod pod-subpath-test-dynamicpv-j8w2 to disappear
Jun 14 16:24:24.334: INFO: Pod pod-subpath-test-dynamicpv-j8w2 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-j8w2
Jun 14 16:24:24.334: INFO: Deleting pod "pod-subpath-test-dynamicpv-j8w2" in namespace "provisioning-8199"
... skipping 39 lines ...
Jun 14 16:23:59.763: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-2448-e2e-scdbjvl
STEP: creating a claim
Jun 14 16:23:59.896: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-mdk6
STEP: Creating a pod to test subpath
Jun 14 16:24:00.299: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-mdk6" in namespace "provisioning-2448" to be "Succeeded or Failed"
Jun 14 16:24:00.432: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 132.985159ms
Jun 14 16:24:02.565: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.26541816s
Jun 14 16:24:04.698: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.398334984s
Jun 14 16:24:06.856: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.55679297s
Jun 14 16:24:08.989: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.689071316s
Jun 14 16:24:11.121: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.82183649s
Jun 14 16:24:13.254: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 12.95465802s
Jun 14 16:24:15.387: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 15.087543704s
Jun 14 16:24:17.519: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 17.219418348s
Jun 14 16:24:19.652: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 19.352323005s
Jun 14 16:24:21.784: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Pending", Reason="", readiness=false. Elapsed: 21.484337938s
Jun 14 16:24:23.917: INFO: Pod "pod-subpath-test-dynamicpv-mdk6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.617182547s
STEP: Saw pod success
Jun 14 16:24:23.917: INFO: Pod "pod-subpath-test-dynamicpv-mdk6" satisfied condition "Succeeded or Failed"
Jun 14 16:24:24.048: INFO: Trying to get logs from node ip-172-20-51-223.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-mdk6 container test-container-subpath-dynamicpv-mdk6: <nil>
STEP: delete the pod
Jun 14 16:24:24.332: INFO: Waiting for pod pod-subpath-test-dynamicpv-mdk6 to disappear
Jun 14 16:24:24.464: INFO: Pod pod-subpath-test-dynamicpv-mdk6 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-mdk6
Jun 14 16:24:24.465: INFO: Deleting pod "pod-subpath-test-dynamicpv-mdk6" in namespace "provisioning-2448"
... skipping 87 lines ...
Jun 14 16:22:53.079: INFO: In creating storage class object and pvc objects for driver - sc: &StorageClass{ObjectMeta:{provisioning-959452g8d      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Provisioner:kubernetes.io/aws-ebs,Parameters:map[string]string{},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-9594    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-959452g8d,VolumeMode:nil,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}, src-pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-9594    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-959452g8d,VolumeMode:nil,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}
STEP: Creating a StorageClass
STEP: creating claim=&PersistentVolumeClaim{ObjectMeta:{ pvc- provisioning-9594    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] []  []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-959452g8d,VolumeMode:nil,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}
STEP: creating a pod referring to the class=&StorageClass{ObjectMeta:{provisioning-959452g8d    e4b93968-37d9-4c45-8a2d-539217a20276 2462 0 2021-06-14 16:22:53 +0000 UTC <nil> <nil> map[] map[] [] []  [{e2e-kubernetes.test Update storage.k8s.io/v1 2021-06-14 16:22:53 +0000 UTC FieldsV1 {"f:mountOptions":{},"f:provisioner":{},"f:reclaimPolicy":{},"f:volumeBindingMode":{}}}]},Provisioner:kubernetes.io/aws-ebs,Parameters:map[string]string{},ReclaimPolicy:*Delete,MountOptions:[debug nouid32],AllowVolumeExpansion:nil,VolumeBindingMode:*WaitForFirstConsumer,AllowedTopologies:[]TopologySelectorTerm{},} claim=&PersistentVolumeClaim{ObjectMeta:{pvc-7tgt2 pvc- provisioning-9594  d29aa15a-c637-4ac3-9508-0aa5bd6bb5af 2482 0 2021-06-14 16:22:53 +0000 UTC <nil> <nil> map[] map[] [] [kubernetes.io/pvc-protection]  [{e2e-kubernetes.test Update v1 2021-06-14 16:22:53 +0000 UTC FieldsV1 {"f:metadata":{"f:generateName":{}},"f:spec":{"f:accessModes":{},"f:resources":{"f:requests":{".":{},"f:storage":{}}},"f:storageClassName":{},"f:volumeMode":{}}}}]},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1073741824 0} {<nil>} 1Gi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*provisioning-959452g8d,VolumeMode:*Filesystem,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:Pending,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},}
STEP: Deleting pod pod-775ead55-0a87-400b-887f-de4d9c29ad2e in namespace provisioning-9594
STEP: checking the created volume is writable on node {Name: Selector:map[] Affinity:nil}
Jun 14 16:23:16.532: INFO: Waiting up to 15m0s for pod "pvc-volume-tester-writer-d86s5" in namespace "provisioning-9594" to be "Succeeded or Failed"
Jun 14 16:23:16.667: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 134.355976ms
Jun 14 16:23:18.801: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.268423578s
Jun 14 16:23:20.932: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.399598383s
Jun 14 16:23:23.065: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.533130031s
Jun 14 16:23:25.198: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.665187729s
Jun 14 16:23:27.329: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.796505954s
... skipping 25 lines ...
Jun 14 16:24:22.772: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.240074613s
Jun 14 16:24:24.906: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.373528618s
Jun 14 16:24:27.068: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.536130719s
Jun 14 16:24:29.201: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.668627736s
Jun 14 16:24:31.338: INFO: Pod "pvc-volume-tester-writer-d86s5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m14.805313865s
STEP: Saw pod success
Jun 14 16:24:31.338: INFO: Pod "pvc-volume-tester-writer-d86s5" satisfied condition "Succeeded or Failed"
Jun 14 16:24:31.604: INFO: Pod pvc-volume-tester-writer-d86s5 has the following logs: 
Jun 14 16:24:31.604: INFO: Deleting pod "pvc-volume-tester-writer-d86s5" in namespace "provisioning-9594"
Jun 14 16:24:31.752: INFO: Wait up to 5m0s for pod "pvc-volume-tester-writer-d86s5" to be fully deleted
STEP: checking the created volume has the correct mount options, is readable and retains data on the same node "ip-172-20-51-223.ap-northeast-1.compute.internal"
Jun 14 16:24:32.286: INFO: Waiting up to 15m0s for pod "pvc-volume-tester-reader-h85dk" in namespace "provisioning-9594" to be "Succeeded or Failed"
Jun 14 16:24:32.420: INFO: Pod "pvc-volume-tester-reader-h85dk": Phase="Pending", Reason="", readiness=false. Elapsed: 133.595804ms
Jun 14 16:24:34.552: INFO: Pod "pvc-volume-tester-reader-h85dk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.265392533s
Jun 14 16:24:36.683: INFO: Pod "pvc-volume-tester-reader-h85dk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.396892443s
Jun 14 16:24:38.816: INFO: Pod "pvc-volume-tester-reader-h85dk": Phase="Pending", Reason="", readiness=false. Elapsed: 6.529338265s
Jun 14 16:24:40.948: INFO: Pod "pvc-volume-tester-reader-h85dk": Phase="Pending", Reason="", readiness=false. Elapsed: 8.661662983s
Jun 14 16:24:43.081: INFO: Pod "pvc-volume-tester-reader-h85dk": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.794775243s
STEP: Saw pod success
Jun 14 16:24:43.081: INFO: Pod "pvc-volume-tester-reader-h85dk" satisfied condition "Succeeded or Failed"
Jun 14 16:24:43.352: INFO: Pod pvc-volume-tester-reader-h85dk has the following logs: hello world

Jun 14 16:24:43.352: INFO: Deleting pod "pvc-volume-tester-reader-h85dk" in namespace "provisioning-9594"
Jun 14 16:24:43.521: INFO: Wait up to 5m0s for pod "pvc-volume-tester-reader-h85dk" to be fully deleted
Jun 14 16:24:43.652: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [pvc-7tgt2] to have phase Bound
Jun 14 16:24:43.784: INFO: PersistentVolumeClaim pvc-7tgt2 found and phase=Bound (131.575188ms)
... skipping 30 lines ...
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:24:30.343: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278
Jun 14 16:24:30.982: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:24:30.982: INFO: Creating resource for dynamic PV
Jun 14 16:24:30.982: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-70957744n
STEP: creating a claim
Jun 14 16:24:31.132: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-fz29
STEP: Checking for subpath error in container status
Jun 14 16:24:47.792: INFO: Deleting pod "pod-subpath-test-dynamicpv-fz29" in namespace "provisioning-7095"
Jun 14 16:24:47.920: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-fz29" to be fully deleted
STEP: Deleting pod
Jun 14 16:24:56.176: INFO: Deleting pod "pod-subpath-test-dynamicpv-fz29" in namespace "provisioning-7095"
STEP: Deleting pvc
Jun 14 16:24:56.559: INFO: Deleting PersistentVolumeClaim "awsx7vq5"
... skipping 13 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Jun 14 16:25:12.597: INFO: Driver supports dynamic provisioning, skipping PreprovisionedPV pattern
[AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
... skipping 301 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Inline-volume (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267

    Driver "ebs.csi.aws.com" does not support volume type "InlineVolume" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 121 lines ...
Jun 14 16:24:26.938: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-1668-e2e-sc97j42
STEP: creating a claim
Jun 14 16:24:27.085: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-mf9n
STEP: Creating a pod to test multi_subpath
Jun 14 16:24:27.484: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-mf9n" in namespace "provisioning-1668" to be "Succeeded or Failed"
Jun 14 16:24:27.614: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Pending", Reason="", readiness=false. Elapsed: 129.615928ms
Jun 14 16:24:29.745: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Pending", Reason="", readiness=false. Elapsed: 2.26059761s
Jun 14 16:24:31.875: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Pending", Reason="", readiness=false. Elapsed: 4.391351628s
Jun 14 16:24:34.007: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Pending", Reason="", readiness=false. Elapsed: 6.523571648s
Jun 14 16:24:36.138: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Pending", Reason="", readiness=false. Elapsed: 8.65376485s
Jun 14 16:24:38.268: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Pending", Reason="", readiness=false. Elapsed: 10.783652693s
Jun 14 16:24:40.399: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Pending", Reason="", readiness=false. Elapsed: 12.914867927s
Jun 14 16:24:42.530: INFO: Pod "pod-subpath-test-dynamicpv-mf9n": Phase="Succeeded", Reason="", readiness=false. Elapsed: 15.045756255s
STEP: Saw pod success
Jun 14 16:24:42.532: INFO: Pod "pod-subpath-test-dynamicpv-mf9n" satisfied condition "Succeeded or Failed"
Jun 14 16:24:42.661: INFO: Trying to get logs from node ip-172-20-56-37.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-mf9n container test-container-subpath-dynamicpv-mf9n: <nil>
STEP: delete the pod
Jun 14 16:24:42.932: INFO: Waiting for pod pod-subpath-test-dynamicpv-mf9n to disappear
Jun 14 16:24:43.061: INFO: Pod pod-subpath-test-dynamicpv-mf9n no longer exists
STEP: Deleting pod
Jun 14 16:24:43.061: INFO: Deleting pod "pod-subpath-test-dynamicpv-mf9n" in namespace "provisioning-1668"
... skipping 113 lines ...
Jun 14 16:24:57.408: INFO: Creating resource for dynamic PV
Jun 14 16:24:57.408: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} 
STEP: creating a StorageClass volume-expand-4224-e2e-scfvq72
STEP: creating a claim
STEP: Expanding non-expandable pvc
Jun 14 16:24:57.787: INFO: currentPvcSize {{1073741824 0} {<nil>} 1Gi BinarySI}, newSize {{2147483648 0} {<nil>}  BinarySI}
Jun 14 16:24:58.043: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:00.295: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:02.296: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:04.291: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:06.291: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:08.291: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:10.291: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:12.295: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:14.297: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:16.295: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:18.297: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:20.299: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:22.297: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:24.298: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:26.296: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:28.303: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-4224-e2e-scfvq72",
  	... // 2 identical fields
  }

Jun 14 16:25:28.556: INFO: Error updating pvc ebs.csi.aws.com5dhv9: PersistentVolumeClaim "ebs.csi.aws.com5dhv9" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 277 lines ...
Jun 14 16:24:10.022: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-2102-e2e-scwpw6f
STEP: creating a claim
Jun 14 16:24:10.160: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-lbc2
STEP: Creating a pod to test atomic-volume-subpath
Jun 14 16:24:10.551: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-lbc2" in namespace "provisioning-2102" to be "Succeeded or Failed"
Jun 14 16:24:10.679: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Pending", Reason="", readiness=false. Elapsed: 128.62089ms
Jun 14 16:24:12.809: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.258185796s
Jun 14 16:24:14.939: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.388232735s
Jun 14 16:24:17.069: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.518064937s
Jun 14 16:24:19.198: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.647763514s
Jun 14 16:24:21.328: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.776859922s
... skipping 11 lines ...
Jun 14 16:24:46.886: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Running", Reason="", readiness=true. Elapsed: 36.335264945s
Jun 14 16:24:49.016: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Running", Reason="", readiness=true. Elapsed: 38.465286334s
Jun 14 16:24:51.145: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Running", Reason="", readiness=true. Elapsed: 40.594267729s
Jun 14 16:24:53.276: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Running", Reason="", readiness=true. Elapsed: 42.724940237s
Jun 14 16:24:55.406: INFO: Pod "pod-subpath-test-dynamicpv-lbc2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 44.855495928s
STEP: Saw pod success
Jun 14 16:24:55.406: INFO: Pod "pod-subpath-test-dynamicpv-lbc2" satisfied condition "Succeeded or Failed"
Jun 14 16:24:55.536: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-lbc2 container test-container-subpath-dynamicpv-lbc2: <nil>
STEP: delete the pod
Jun 14 16:24:55.809: INFO: Waiting for pod pod-subpath-test-dynamicpv-lbc2 to disappear
Jun 14 16:24:55.940: INFO: Pod pod-subpath-test-dynamicpv-lbc2 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-lbc2
Jun 14 16:24:55.940: INFO: Deleting pod "pod-subpath-test-dynamicpv-lbc2" in namespace "provisioning-2102"
... skipping 228 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256

      Distro debian doesn't support ntfs -- skipping

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:127
------------------------------
... skipping 36 lines ...
Jun 14 16:24:30.449: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-6553-e2e-sc24852
STEP: creating a claim
Jun 14 16:24:30.581: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-bs8q
STEP: Creating a pod to test subpath
Jun 14 16:24:30.988: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-bs8q" in namespace "provisioning-6553" to be "Succeeded or Failed"
Jun 14 16:24:31.146: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 157.604367ms
Jun 14 16:24:33.276: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.288049376s
Jun 14 16:24:35.408: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.419193606s
Jun 14 16:24:37.538: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.549872411s
Jun 14 16:24:39.669: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.680272461s
Jun 14 16:24:41.799: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.810967817s
... skipping 2 lines ...
Jun 14 16:24:48.192: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 17.203922676s
Jun 14 16:24:50.322: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 19.333873412s
Jun 14 16:24:52.456: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 21.467095471s
Jun 14 16:24:54.586: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Pending", Reason="", readiness=false. Elapsed: 23.59745244s
Jun 14 16:24:56.720: INFO: Pod "pod-subpath-test-dynamicpv-bs8q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 25.731265625s
STEP: Saw pod success
Jun 14 16:24:56.720: INFO: Pod "pod-subpath-test-dynamicpv-bs8q" satisfied condition "Succeeded or Failed"
Jun 14 16:24:56.850: INFO: Trying to get logs from node ip-172-20-51-223.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-bs8q container test-container-subpath-dynamicpv-bs8q: <nil>
STEP: delete the pod
Jun 14 16:24:57.124: INFO: Waiting for pod pod-subpath-test-dynamicpv-bs8q to disappear
Jun 14 16:24:57.254: INFO: Pod pod-subpath-test-dynamicpv-bs8q no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-bs8q
Jun 14 16:24:57.254: INFO: Deleting pod "pod-subpath-test-dynamicpv-bs8q" in namespace "provisioning-6553"
... skipping 61 lines ...
Jun 14 16:24:26.499: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-4445pgb5
STEP: creating a claim
Jun 14 16:24:26.625: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-fc97
STEP: Creating a pod to test subpath
Jun 14 16:24:27.045: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-fc97" in namespace "provisioning-444" to be "Succeeded or Failed"
Jun 14 16:24:27.180: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 134.969196ms
Jun 14 16:24:29.306: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 2.260879825s
Jun 14 16:24:31.432: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 4.386934681s
Jun 14 16:24:33.559: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 6.513613762s
Jun 14 16:24:35.684: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 8.639156754s
Jun 14 16:24:37.811: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 10.765597267s
... skipping 9 lines ...
Jun 14 16:24:59.074: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 32.028463213s
Jun 14 16:25:01.200: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 34.154883767s
Jun 14 16:25:03.325: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 36.280123421s
Jun 14 16:25:05.451: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Pending", Reason="", readiness=false. Elapsed: 38.406192773s
Jun 14 16:25:07.577: INFO: Pod "pod-subpath-test-dynamicpv-fc97": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.532102095s
STEP: Saw pod success
Jun 14 16:25:07.577: INFO: Pod "pod-subpath-test-dynamicpv-fc97" satisfied condition "Succeeded or Failed"
Jun 14 16:25:07.702: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-fc97 container test-container-volume-dynamicpv-fc97: <nil>
STEP: delete the pod
Jun 14 16:25:07.961: INFO: Waiting for pod pod-subpath-test-dynamicpv-fc97 to disappear
Jun 14 16:25:08.086: INFO: Pod pod-subpath-test-dynamicpv-fc97 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-fc97
Jun 14 16:25:08.086: INFO: Deleting pod "pod-subpath-test-dynamicpv-fc97" in namespace "provisioning-444"
... skipping 323 lines ...
[BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:25:44.410: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename topology
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Jun 14 16:25:45.168: INFO: found topology map[topology.kubernetes.io/zone:ap-northeast-1a]
Jun 14 16:25:45.168: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:25:45.168: INFO: Not enough topologies in cluster -- skipping
STEP: Deleting pvc
STEP: Deleting sc
... skipping 7 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (immediate binding)] topology
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail to schedule a pod which has topologies that conflict with AllowedTopologies [It]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

      Not enough topologies in cluster -- skipping

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:199
------------------------------
... skipping 166 lines ...
STEP: Deleting pod hostexec-ip-172-20-51-223.ap-northeast-1.compute.internal-hfkx7 in namespace volumemode-1285
Jun 14 16:25:30.160: INFO: Deleting pod "pod-513d21af-a3f5-4ad9-897e-e5c1dfd07fa4" in namespace "volumemode-1285"
Jun 14 16:25:30.286: INFO: Wait up to 5m0s for pod "pod-513d21af-a3f5-4ad9-897e-e5c1dfd07fa4" to be fully deleted
STEP: Deleting pv and pvc
Jun 14 16:25:38.532: INFO: Deleting PersistentVolumeClaim "pvc-mkf8r"
Jun 14 16:25:38.658: INFO: Deleting PersistentVolume "aws-fk8r4"
Jun 14 16:25:39.175: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-03c25708801c3df78", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-03c25708801c3df78 is currently attached to i-07389f3062fba0924
	status code: 400, request id: f4b6521c-9f76-40e3-8329-df601d831654
Jun 14 16:25:44.923: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-03c25708801c3df78", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-03c25708801c3df78 is currently attached to i-07389f3062fba0924
	status code: 400, request id: 99f50aee-ca13-40f8-b89a-db5760a188c8
Jun 14 16:25:50.605: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-03c25708801c3df78".
[AfterEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:25:50.605: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volumemode-1285" for this suite.
... skipping 33 lines ...
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:25:32.966: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename volumemode
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to use a volume in a pod with mismatched mode [Slow]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
Jun 14 16:25:33.612: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:25:33.612: INFO: Creating resource for dynamic PV
Jun 14 16:25:33.612: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass volumemode-4978whbhd
STEP: creating a claim
STEP: Creating pod
STEP: Waiting for the pod to fail
Jun 14 16:25:40.522: INFO: Deleting pod "pod-b8f0bf3a-d87f-460c-b67b-1205d41e5747" in namespace "volumemode-4978"
Jun 14 16:25:40.655: INFO: Wait up to 5m0s for pod "pod-b8f0bf3a-d87f-460c-b67b-1205d41e5747" to be fully deleted
STEP: Deleting pvc
Jun 14 16:25:47.171: INFO: Deleting PersistentVolumeClaim "aws68ltb"
Jun 14 16:25:47.306: INFO: Waiting up to 5m0s for PersistentVolume pvc-a9965415-f1f3-4e74-8532-3f0f5293e350 to get deleted
Jun 14 16:25:47.439: INFO: PersistentVolume pvc-a9965415-f1f3-4e74-8532-3f0f5293e350 found and phase=Released (133.129008ms)
... skipping 10 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (block volmode)] volumeMode
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail to use a volume in a pod with mismatched mode [Slow]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Jun 14 16:25:58.082: INFO: Driver ebs.csi.aws.com doesn't support ntfs -- skipping
[AfterEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
... skipping 40 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Inline-volume (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278

    Driver "ebs.csi.aws.com" does not support volume type "InlineVolume" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 226 lines ...
Jun 14 16:25:29.872: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-2603vjj9f
STEP: creating a claim
Jun 14 16:25:29.998: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-gzg4
STEP: Creating a pod to test multi_subpath
Jun 14 16:25:30.380: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-gzg4" in namespace "provisioning-2603" to be "Succeeded or Failed"
Jun 14 16:25:30.508: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 128.256685ms
Jun 14 16:25:32.634: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.254199304s
Jun 14 16:25:34.760: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.380185664s
Jun 14 16:25:36.888: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.507547823s
Jun 14 16:25:39.019: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.638540588s
Jun 14 16:25:41.145: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.764641113s
Jun 14 16:25:43.272: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 12.891575219s
Jun 14 16:25:45.398: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 15.01836094s
Jun 14 16:25:47.525: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 17.144723066s
Jun 14 16:25:49.651: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 19.270944978s
Jun 14 16:25:51.777: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Pending", Reason="", readiness=false. Elapsed: 21.396996913s
Jun 14 16:25:53.903: INFO: Pod "pod-subpath-test-dynamicpv-gzg4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.523393419s
STEP: Saw pod success
Jun 14 16:25:53.903: INFO: Pod "pod-subpath-test-dynamicpv-gzg4" satisfied condition "Succeeded or Failed"
Jun 14 16:25:54.032: INFO: Trying to get logs from node ip-172-20-51-223.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-gzg4 container test-container-subpath-dynamicpv-gzg4: <nil>
STEP: delete the pod
Jun 14 16:25:54.298: INFO: Waiting for pod pod-subpath-test-dynamicpv-gzg4 to disappear
Jun 14 16:25:54.426: INFO: Pod pod-subpath-test-dynamicpv-gzg4 no longer exists
STEP: Deleting pod
Jun 14 16:25:54.426: INFO: Deleting pod "pod-subpath-test-dynamicpv-gzg4" in namespace "provisioning-2603"
... skipping 34 lines ...
Jun 14 16:25:36.386: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
STEP: creating a test aws volume
Jun 14 16:25:37.319: INFO: Successfully created a new PD: "aws://ap-northeast-1a/vol-0e9f56fb8d69a6adc".
Jun 14 16:25:37.319: INFO: Creating resource for inline volume
STEP: Creating pod exec-volume-test-inlinevolume-7qht
STEP: Creating a pod to test exec-volume-test
Jun 14 16:25:37.447: INFO: Waiting up to 5m0s for pod "exec-volume-test-inlinevolume-7qht" in namespace "volume-4095" to be "Succeeded or Failed"
Jun 14 16:25:37.573: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 125.848451ms
Jun 14 16:25:39.701: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 2.254047624s
Jun 14 16:25:41.827: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 4.379938917s
Jun 14 16:25:43.954: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 6.507179081s
Jun 14 16:25:46.080: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 8.633411506s
Jun 14 16:25:48.207: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 10.760019255s
Jun 14 16:25:50.335: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 12.888442022s
Jun 14 16:25:52.462: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Pending", Reason="", readiness=false. Elapsed: 15.01483353s
Jun 14 16:25:54.590: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Running", Reason="", readiness=true. Elapsed: 17.143112916s
Jun 14 16:25:56.718: INFO: Pod "exec-volume-test-inlinevolume-7qht": Phase="Succeeded", Reason="", readiness=false. Elapsed: 19.271332103s
STEP: Saw pod success
Jun 14 16:25:56.718: INFO: Pod "exec-volume-test-inlinevolume-7qht" satisfied condition "Succeeded or Failed"
Jun 14 16:25:56.843: INFO: Trying to get logs from node ip-172-20-56-37.ap-northeast-1.compute.internal pod exec-volume-test-inlinevolume-7qht container exec-container-inlinevolume-7qht: <nil>
STEP: delete the pod
Jun 14 16:25:57.106: INFO: Waiting for pod exec-volume-test-inlinevolume-7qht to disappear
Jun 14 16:25:57.233: INFO: Pod exec-volume-test-inlinevolume-7qht no longer exists
STEP: Deleting pod exec-volume-test-inlinevolume-7qht
Jun 14 16:25:57.233: INFO: Deleting pod "exec-volume-test-inlinevolume-7qht" in namespace "volume-4095"
Jun 14 16:25:57.616: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0e9f56fb8d69a6adc", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0e9f56fb8d69a6adc is currently attached to i-02f6d88db741a2194
	status code: 400, request id: 51d1c836-9e6b-4a1c-94e2-9e2920dad99c
Jun 14 16:26:03.343: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0e9f56fb8d69a6adc", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0e9f56fb8d69a6adc is currently attached to i-02f6d88db741a2194
	status code: 400, request id: 53699976-3524-4857-ba97-a966dca77f79
Jun 14 16:26:09.126: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0e9f56fb8d69a6adc", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0e9f56fb8d69a6adc is currently attached to i-02f6d88db741a2194
	status code: 400, request id: 5bc62586-fbbb-4341-b70f-7671b896b340
Jun 14 16:26:14.891: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-0e9f56fb8d69a6adc".
[AfterEach] [Testpattern: Inline-volume (default fs)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:26:14.891: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-4095" for this suite.
... skipping 235 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Inline-volume (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256

      Driver supports dynamic provisioning, skipping InlineVolume pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 320 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Inline-volume (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267

      Driver supports dynamic provisioning, skipping InlineVolume pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 25 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Inline-volume (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240

    Driver "ebs.csi.aws.com" does not support volume type "InlineVolume" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 6 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.000 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (delayed binding)] topology
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver "ebs.csi.aws.com" does not support topology - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:92
------------------------------
... skipping 222 lines ...
Jun 14 16:26:49.095: INFO: AfterEach: Cleaning up test resources


S [SKIPPING] in Spec Setup (BeforeEach) [0.915 seconds]
[sig-storage] PersistentVolumes:vsphere [Feature:vsphere]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23
  should test that deleting a PVC before the pod does not cause pod deletion to fail on vsphere volume detach [BeforeEach]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/vsphere/persistent_volumes-vsphere.go:148

  Only supported for providers [vsphere] (not aws)

  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/vsphere/persistent_volumes-vsphere.go:64
------------------------------
... skipping 4 lines ...
[BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:26:10.826: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename volumemode
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to use a volume in a pod with mismatched mode [Slow]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
Jun 14 16:26:11.455: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
STEP: creating a test aws volume
Jun 14 16:26:12.309: INFO: Successfully created a new PD: "aws://ap-northeast-1a/vol-01b46f1151cd8e077".
Jun 14 16:26:12.309: INFO: Creating resource for pre-provisioned PV
Jun 14 16:26:12.309: INFO: Creating PVC and PV
... skipping 6 lines ...
Jun 14 16:26:19.154: INFO: PersistentVolumeClaim pvc-mjn5l found but phase is Pending instead of Bound.
Jun 14 16:26:21.281: INFO: PersistentVolumeClaim pvc-mjn5l found but phase is Pending instead of Bound.
Jun 14 16:26:23.406: INFO: PersistentVolumeClaim pvc-mjn5l found and phase=Bound (10.757425692s)
Jun 14 16:26:23.406: INFO: Waiting up to 3m0s for PersistentVolume aws-trtkd to have phase Bound
Jun 14 16:26:23.532: INFO: PersistentVolume aws-trtkd found and phase=Bound (125.675708ms)
STEP: Creating pod
STEP: Waiting for the pod to fail
Jun 14 16:26:26.287: INFO: Deleting pod "pod-f10564b3-f036-4633-a65a-d0e3c7a438a1" in namespace "volumemode-8415"
Jun 14 16:26:26.414: INFO: Wait up to 5m0s for pod "pod-f10564b3-f036-4633-a65a-d0e3c7a438a1" to be fully deleted
STEP: Deleting pv and pvc
Jun 14 16:26:36.665: INFO: Deleting PersistentVolumeClaim "pvc-mjn5l"
Jun 14 16:26:36.791: INFO: Deleting PersistentVolume "aws-trtkd"
Jun 14 16:26:37.323: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-01b46f1151cd8e077", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-01b46f1151cd8e077 is currently attached to i-02f6d88db741a2194
	status code: 400, request id: 98614243-6985-44ee-9371-17800eeac2b8
Jun 14 16:26:43.189: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-01b46f1151cd8e077", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-01b46f1151cd8e077 is currently attached to i-02f6d88db741a2194
	status code: 400, request id: 5d19031d-2ea0-4376-bfb2-29e7677b6c52
Jun 14 16:26:48.927: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-01b46f1151cd8e077".
[AfterEach] [Testpattern: Pre-provisioned PV (block volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:26:48.927: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volumemode-8415" for this suite.
... skipping 3 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Pre-provisioned PV (block volmode)] volumeMode
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail to use a volume in a pod with mismatched mode [Slow]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
------------------------------
[BeforeEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable-stress[Feature:VolumeSnapshotDataSource]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Jun 14 16:26:49.182: INFO: Driver aws doesn't specify snapshot stress test options -- skipping
[AfterEach] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable-stress[Feature:VolumeSnapshotDataSource]
... skipping 207 lines ...
Jun 14 16:26:05.175: INFO: PersistentVolumeClaim pvc-kwj9x found but phase is Pending instead of Bound.
Jun 14 16:26:07.302: INFO: PersistentVolumeClaim pvc-kwj9x found and phase=Bound (14.998756736s)
Jun 14 16:26:07.302: INFO: Waiting up to 3m0s for PersistentVolume aws-bf667 to have phase Bound
Jun 14 16:26:07.425: INFO: PersistentVolume aws-bf667 found and phase=Bound (122.976614ms)
STEP: Creating pod exec-volume-test-preprovisionedpv-c5b7
STEP: Creating a pod to test exec-volume-test
Jun 14 16:26:07.794: INFO: Waiting up to 5m0s for pod "exec-volume-test-preprovisionedpv-c5b7" in namespace "volume-8469" to be "Succeeded or Failed"
Jun 14 16:26:07.930: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 135.176349ms
Jun 14 16:26:10.053: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.258814517s
Jun 14 16:26:12.177: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.382144372s
Jun 14 16:26:14.301: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.506180563s
Jun 14 16:26:16.426: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.631032039s
Jun 14 16:26:18.553: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.758176718s
... skipping 7 lines ...
Jun 14 16:26:35.549: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 27.754874184s
Jun 14 16:26:37.682: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 29.887180876s
Jun 14 16:26:39.805: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Pending", Reason="", readiness=false. Elapsed: 32.010196956s
Jun 14 16:26:41.928: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Running", Reason="", readiness=true. Elapsed: 34.133554593s
Jun 14 16:26:44.052: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 36.257359559s
STEP: Saw pod success
Jun 14 16:26:44.052: INFO: Pod "exec-volume-test-preprovisionedpv-c5b7" satisfied condition "Succeeded or Failed"
Jun 14 16:26:44.175: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod exec-volume-test-preprovisionedpv-c5b7 container exec-container-preprovisionedpv-c5b7: <nil>
STEP: delete the pod
Jun 14 16:26:44.451: INFO: Waiting for pod exec-volume-test-preprovisionedpv-c5b7 to disappear
Jun 14 16:26:44.575: INFO: Pod exec-volume-test-preprovisionedpv-c5b7 no longer exists
STEP: Deleting pod exec-volume-test-preprovisionedpv-c5b7
Jun 14 16:26:44.575: INFO: Deleting pod "exec-volume-test-preprovisionedpv-c5b7" in namespace "volume-8469"
STEP: Deleting pv and pvc
Jun 14 16:26:44.698: INFO: Deleting PersistentVolumeClaim "pvc-kwj9x"
Jun 14 16:26:44.824: INFO: Deleting PersistentVolume "aws-bf667"
Jun 14 16:26:45.309: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0639756dd9d512e84", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0639756dd9d512e84 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: ef500bc3-e2bc-4ded-8317-82daa3fd728b
Jun 14 16:26:51.101: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0639756dd9d512e84", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0639756dd9d512e84 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: d2417c24-0c37-4cb5-8581-ee1e47ba54b0
Jun 14 16:26:56.874: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-0639756dd9d512e84".
[AfterEach] [Testpattern: Pre-provisioned PV (default fs)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:26:56.874: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-8469" for this suite.
... skipping 208 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Pre-provisioned PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267

      Driver supports dynamic provisioning, skipping PreprovisionedPV pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 208 lines ...
Jun 14 16:26:17.559: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-7108wlps
STEP: creating a claim
Jun 14 16:26:17.685: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-bd8v
STEP: Creating a pod to test subpath
Jun 14 16:26:18.072: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-bd8v" in namespace "provisioning-710" to be "Succeeded or Failed"
Jun 14 16:26:18.199: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 126.887761ms
Jun 14 16:26:20.325: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 2.253143456s
Jun 14 16:26:22.451: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 4.378846705s
Jun 14 16:26:24.580: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 6.507835462s
Jun 14 16:26:26.705: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 8.633583431s
Jun 14 16:26:28.831: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 10.759063402s
... skipping 3 lines ...
Jun 14 16:26:37.363: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 19.291200287s
Jun 14 16:26:39.489: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 21.417207358s
Jun 14 16:26:41.615: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 23.542742095s
Jun 14 16:26:43.742: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Pending", Reason="", readiness=false. Elapsed: 25.66989047s
Jun 14 16:26:45.867: INFO: Pod "pod-subpath-test-dynamicpv-bd8v": Phase="Succeeded", Reason="", readiness=false. Elapsed: 27.795628956s
STEP: Saw pod success
Jun 14 16:26:45.868: INFO: Pod "pod-subpath-test-dynamicpv-bd8v" satisfied condition "Succeeded or Failed"
Jun 14 16:26:45.992: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-bd8v container test-container-volume-dynamicpv-bd8v: <nil>
STEP: delete the pod
Jun 14 16:26:46.277: INFO: Waiting for pod pod-subpath-test-dynamicpv-bd8v to disappear
Jun 14 16:26:46.402: INFO: Pod pod-subpath-test-dynamicpv-bd8v no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-bd8v
Jun 14 16:26:46.402: INFO: Deleting pod "pod-subpath-test-dynamicpv-bd8v" in namespace "provisioning-710"
... skipping 336 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Pre-provisioned PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267

    Driver "ebs.csi.aws.com" does not support volume type "PreprovisionedPV" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 11 lines ...
Jun 14 16:26:20.014: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-3828-e2e-sch85d7
STEP: creating a claim
Jun 14 16:26:20.149: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-w97c
STEP: Creating a pod to test subpath
Jun 14 16:26:20.566: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-w97c" in namespace "provisioning-3828" to be "Succeeded or Failed"
Jun 14 16:26:20.698: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 132.17278ms
Jun 14 16:26:22.832: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.266293409s
Jun 14 16:26:24.965: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.399302757s
Jun 14 16:26:27.099: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.532989238s
Jun 14 16:26:29.233: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.666850421s
Jun 14 16:26:31.366: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.799739372s
Jun 14 16:26:33.499: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 12.933323099s
Jun 14 16:26:35.633: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 15.066654195s
Jun 14 16:26:37.766: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 17.199528111s
Jun 14 16:26:39.899: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 19.333124922s
Jun 14 16:26:42.033: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Pending", Reason="", readiness=false. Elapsed: 21.466597516s
Jun 14 16:26:44.165: INFO: Pod "pod-subpath-test-dynamicpv-w97c": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.599349493s
STEP: Saw pod success
Jun 14 16:26:44.165: INFO: Pod "pod-subpath-test-dynamicpv-w97c" satisfied condition "Succeeded or Failed"
Jun 14 16:26:44.298: INFO: Trying to get logs from node ip-172-20-51-223.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-w97c container test-container-volume-dynamicpv-w97c: <nil>
STEP: delete the pod
Jun 14 16:26:44.581: INFO: Waiting for pod pod-subpath-test-dynamicpv-w97c to disappear
Jun 14 16:26:44.716: INFO: Pod pod-subpath-test-dynamicpv-w97c no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-w97c
Jun 14 16:26:44.716: INFO: Deleting pod "pod-subpath-test-dynamicpv-w97c" in namespace "provisioning-3828"
... skipping 177 lines ...
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:26:49.098: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267
Jun 14 16:26:49.745: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:26:49.745: INFO: Creating resource for dynamic PV
Jun 14 16:26:49.745: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-8653bwbxt
STEP: creating a claim
Jun 14 16:26:49.874: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-hvss
STEP: Checking for subpath error in container status
Jun 14 16:27:02.530: INFO: Deleting pod "pod-subpath-test-dynamicpv-hvss" in namespace "provisioning-8653"
Jun 14 16:27:02.663: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-hvss" to be fully deleted
STEP: Deleting pod
Jun 14 16:27:12.922: INFO: Deleting pod "pod-subpath-test-dynamicpv-hvss" in namespace "provisioning-8653"
STEP: Deleting pvc
Jun 14 16:27:13.311: INFO: Deleting PersistentVolumeClaim "awsm2587"
... skipping 12 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267
------------------------------
[BeforeEach] [sig-storage] vcp-performance [Feature:vsphere]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:27:24.226: INFO: >>> kubeConfig: /root/.kube/config
... skipping 252 lines ...
Jun 14 16:26:59.755: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(aws) supported size:{ 1Gi} 
STEP: creating a StorageClass volume-expand-5897q65cl
STEP: creating a claim
Jun 14 16:26:59.883: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Expanding non-expandable pvc
Jun 14 16:27:00.150: INFO: currentPvcSize {{1073741824 0} {<nil>} 1Gi BinarySI}, newSize {{2147483648 0} {<nil>}  BinarySI}
Jun 14 16:27:00.408: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:02.668: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:04.664: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:06.664: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:08.672: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:10.665: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:12.665: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:14.666: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:16.665: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:18.665: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:20.666: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:22.671: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:24.665: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:26.665: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:28.665: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:30.669: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-5897q65cl",
  	... // 2 identical fields
  }

Jun 14 16:27:30.926: INFO: Error updating pvc aws658fz: PersistentVolumeClaim "aws658fz" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 36 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278

    Driver ebs.csi.aws.com doesn't support ntfs -- skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:121
------------------------------
... skipping 201 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Pre-provisioned PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278

      Driver supports dynamic provisioning, skipping PreprovisionedPV pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 147 lines ...
Jun 14 16:25:49.953: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-2009phdj2
STEP: creating a claim
Jun 14 16:25:50.083: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-jxg9
STEP: Creating a pod to test subpath
Jun 14 16:25:50.475: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-jxg9" in namespace "provisioning-2009" to be "Succeeded or Failed"
Jun 14 16:25:50.604: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 128.072957ms
Jun 14 16:25:52.732: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.257015112s
Jun 14 16:25:54.863: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.387964209s
Jun 14 16:25:56.992: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.517020927s
Jun 14 16:25:59.121: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.645538845s
Jun 14 16:26:01.250: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.77428679s
Jun 14 16:26:03.378: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 12.90287988s
Jun 14 16:26:05.508: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 15.032263894s
Jun 14 16:26:07.639: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 17.163066702s
STEP: Saw pod success
Jun 14 16:26:07.639: INFO: Pod "pod-subpath-test-dynamicpv-jxg9" satisfied condition "Succeeded or Failed"
Jun 14 16:26:07.769: INFO: Trying to get logs from node ip-172-20-56-37.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-jxg9 container test-container-subpath-dynamicpv-jxg9: <nil>
STEP: delete the pod
Jun 14 16:26:08.040: INFO: Waiting for pod pod-subpath-test-dynamicpv-jxg9 to disappear
Jun 14 16:26:08.168: INFO: Pod pod-subpath-test-dynamicpv-jxg9 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-jxg9
Jun 14 16:26:08.168: INFO: Deleting pod "pod-subpath-test-dynamicpv-jxg9" in namespace "provisioning-2009"
STEP: Creating pod pod-subpath-test-dynamicpv-jxg9
STEP: Creating a pod to test subpath
Jun 14 16:26:08.428: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-jxg9" in namespace "provisioning-2009" to be "Succeeded or Failed"
Jun 14 16:26:08.556: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 128.433177ms
Jun 14 16:26:10.686: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.257619334s
Jun 14 16:26:12.816: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.387962994s
Jun 14 16:26:14.945: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.516892213s
Jun 14 16:26:17.075: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.647002322s
Jun 14 16:26:19.204: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.775835112s
... skipping 22 lines ...
Jun 14 16:27:08.182: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 59.754062535s
Jun 14 16:27:10.311: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 1m1.882952724s
Jun 14 16:27:12.443: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.014750658s
Jun 14 16:27:14.572: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.143663217s
Jun 14 16:27:16.700: INFO: Pod "pod-subpath-test-dynamicpv-jxg9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m8.272322915s
STEP: Saw pod success
Jun 14 16:27:16.700: INFO: Pod "pod-subpath-test-dynamicpv-jxg9" satisfied condition "Succeeded or Failed"
Jun 14 16:27:16.829: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-jxg9 container test-container-subpath-dynamicpv-jxg9: <nil>
STEP: delete the pod
Jun 14 16:27:17.098: INFO: Waiting for pod pod-subpath-test-dynamicpv-jxg9 to disappear
Jun 14 16:27:17.227: INFO: Pod pod-subpath-test-dynamicpv-jxg9 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-jxg9
Jun 14 16:27:17.227: INFO: Deleting pod "pod-subpath-test-dynamicpv-jxg9" in namespace "provisioning-2009"
... skipping 38 lines ...
Jun 14 16:27:08.713: INFO: Creating resource for dynamic PV
Jun 14 16:27:08.713: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(aws) supported size:{ 1Gi} 
STEP: creating a StorageClass volume-expand-849wv72s
STEP: creating a claim
STEP: Expanding non-expandable pvc
Jun 14 16:27:09.092: INFO: currentPvcSize {{1073741824 0} {<nil>} 1Gi BinarySI}, newSize {{2147483648 0} {<nil>}  BinarySI}
Jun 14 16:27:09.346: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:11.597: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:13.597: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:15.598: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:17.638: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:19.598: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:21.598: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:23.599: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:25.598: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:27.627: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:29.604: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:31.598: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:33.601: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:35.597: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:37.599: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:39.598: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-849wv72s",
  	... // 2 identical fields
  }

Jun 14 16:27:39.849: INFO: Error updating pvc awscvjsh: PersistentVolumeClaim "awscvjsh" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 106 lines ...
Jun 14 16:27:41.380: INFO: AfterEach: Cleaning up test resources


S [SKIPPING] in Spec Setup (BeforeEach) [0.880 seconds]
[sig-storage] PersistentVolumes:vsphere [Feature:vsphere]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/utils/framework.go:23
  should test that deleting the PV before the pod does not cause pod deletion to fail on vsphere volume detach [BeforeEach]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/vsphere/persistent_volumes-vsphere.go:164

  Only supported for providers [vsphere] (not aws)

  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/vsphere/persistent_volumes-vsphere.go:64
------------------------------
... skipping 281 lines ...
Jun 14 16:27:05.101: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-4959ws7nc
STEP: creating a claim
Jun 14 16:27:05.230: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-98xr
STEP: Creating a pod to test subpath
Jun 14 16:27:05.619: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-98xr" in namespace "provisioning-4959" to be "Succeeded or Failed"
Jun 14 16:27:05.747: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 128.10646ms
Jun 14 16:27:07.876: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.257257783s
Jun 14 16:27:10.006: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.387057165s
Jun 14 16:27:12.135: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.516393808s
Jun 14 16:27:14.265: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.646213806s
Jun 14 16:27:16.395: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.776014128s
Jun 14 16:27:18.524: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 12.904991848s
Jun 14 16:27:20.654: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 15.034859703s
Jun 14 16:27:22.783: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 17.16378439s
Jun 14 16:27:24.914: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 19.294670113s
Jun 14 16:27:27.044: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Pending", Reason="", readiness=false. Elapsed: 21.424859943s
Jun 14 16:27:29.173: INFO: Pod "pod-subpath-test-dynamicpv-98xr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.553984583s
STEP: Saw pod success
Jun 14 16:27:29.173: INFO: Pod "pod-subpath-test-dynamicpv-98xr" satisfied condition "Succeeded or Failed"
Jun 14 16:27:29.303: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-98xr container test-container-subpath-dynamicpv-98xr: <nil>
STEP: delete the pod
Jun 14 16:27:29.585: INFO: Waiting for pod pod-subpath-test-dynamicpv-98xr to disappear
Jun 14 16:27:29.713: INFO: Pod pod-subpath-test-dynamicpv-98xr no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-98xr
Jun 14 16:27:29.713: INFO: Deleting pod "pod-subpath-test-dynamicpv-98xr" in namespace "provisioning-4959"
... skipping 72 lines ...
[BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:27:14.026: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename volumemode
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to use a volume in a pod with mismatched mode [Slow]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
Jun 14 16:27:14.666: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:27:14.666: INFO: Creating resource for dynamic PV
Jun 14 16:27:14.666: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass volumemode-4968xf548
STEP: creating a claim
STEP: Creating pod
STEP: Waiting for the pod to fail
Jun 14 16:27:19.569: INFO: Deleting pod "pod-4fab0ba9-9754-4cd4-8ed3-410cea994c8c" in namespace "volumemode-4968"
Jun 14 16:27:19.698: INFO: Wait up to 5m0s for pod "pod-4fab0ba9-9754-4cd4-8ed3-410cea994c8c" to be fully deleted
STEP: Deleting pvc
Jun 14 16:27:32.213: INFO: Deleting PersistentVolumeClaim "awsctkjs"
Jun 14 16:27:32.346: INFO: Waiting up to 5m0s for PersistentVolume pvc-9a511b9a-6fd6-44e6-bc8a-8307ab17dbad to get deleted
Jun 14 16:27:32.475: INFO: PersistentVolume pvc-9a511b9a-6fd6-44e6-bc8a-8307ab17dbad found and phase=Released (129.16556ms)
... skipping 11 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail to use a volume in a pod with mismatched mode [Slow]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
------------------------------
S
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
... skipping 134 lines ...
Jun 14 16:26:56.132: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass volume-1884-e2e-scphjcz
STEP: creating a claim
Jun 14 16:26:56.259: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod exec-volume-test-dynamicpv-8qkw
STEP: Creating a pod to test exec-volume-test
Jun 14 16:26:56.647: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-8qkw" in namespace "volume-1884" to be "Succeeded or Failed"
Jun 14 16:26:56.773: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 126.05082ms
Jun 14 16:26:58.902: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.254188416s
Jun 14 16:27:01.029: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.381952694s
Jun 14 16:27:03.158: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 6.510531717s
Jun 14 16:27:05.285: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 8.63801067s
Jun 14 16:27:07.413: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 10.765975074s
... skipping 3 lines ...
Jun 14 16:27:15.923: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 19.275529431s
Jun 14 16:27:18.049: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 21.402076745s
Jun 14 16:27:20.177: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 23.529387765s
Jun 14 16:27:22.312: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Pending", Reason="", readiness=false. Elapsed: 25.66505684s
Jun 14 16:27:24.441: INFO: Pod "exec-volume-test-dynamicpv-8qkw": Phase="Succeeded", Reason="", readiness=false. Elapsed: 27.793378592s
STEP: Saw pod success
Jun 14 16:27:24.441: INFO: Pod "exec-volume-test-dynamicpv-8qkw" satisfied condition "Succeeded or Failed"
Jun 14 16:27:24.568: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod exec-volume-test-dynamicpv-8qkw container exec-container-dynamicpv-8qkw: <nil>
STEP: delete the pod
Jun 14 16:27:24.834: INFO: Waiting for pod exec-volume-test-dynamicpv-8qkw to disappear
Jun 14 16:27:24.967: INFO: Pod exec-volume-test-dynamicpv-8qkw no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-8qkw
Jun 14 16:27:24.968: INFO: Deleting pod "exec-volume-test-dynamicpv-8qkw" in namespace "volume-1884"
... skipping 356 lines ...
[BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:27:42.279: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename volumemode
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to use a volume in a pod with mismatched mode [Slow]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
Jun 14 16:27:42.909: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
STEP: creating a test aws volume
Jun 14 16:27:43.812: INFO: Successfully created a new PD: "aws://ap-northeast-1a/vol-0b9bc4c4addf33bca".
Jun 14 16:27:43.812: INFO: Creating resource for pre-provisioned PV
Jun 14 16:27:43.812: INFO: Creating PVC and PV
... skipping 5 lines ...
Jun 14 16:27:48.546: INFO: PersistentVolumeClaim pvc-v6b82 found but phase is Pending instead of Bound.
Jun 14 16:27:50.672: INFO: PersistentVolumeClaim pvc-v6b82 found but phase is Pending instead of Bound.
Jun 14 16:27:52.802: INFO: PersistentVolumeClaim pvc-v6b82 found and phase=Bound (8.642877105s)
Jun 14 16:27:52.802: INFO: Waiting up to 3m0s for PersistentVolume aws-qptgq to have phase Bound
Jun 14 16:27:52.927: INFO: PersistentVolume aws-qptgq found and phase=Bound (124.883006ms)
STEP: Creating pod
STEP: Waiting for the pod to fail
Jun 14 16:27:55.688: INFO: Deleting pod "pod-2d196be0-2ad3-487c-87a5-5f6d59c84c83" in namespace "volumemode-2660"
Jun 14 16:27:55.820: INFO: Wait up to 5m0s for pod "pod-2d196be0-2ad3-487c-87a5-5f6d59c84c83" to be fully deleted
STEP: Deleting pv and pvc
Jun 14 16:28:08.077: INFO: Deleting PersistentVolumeClaim "pvc-v6b82"
Jun 14 16:28:08.203: INFO: Deleting PersistentVolume "aws-qptgq"
Jun 14 16:28:08.638: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-0b9bc4c4addf33bca".
... skipping 7 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail to use a volume in a pod with mismatched mode [Slow]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Jun 14 16:28:08.892: INFO: Driver supports dynamic provisioning, skipping PreprovisionedPV pattern
[AfterEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow]
... skipping 156 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Pre-provisioned PV (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240

      Driver supports dynamic provisioning, skipping PreprovisionedPV pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 31 lines ...
Jun 14 16:27:28.071: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-555fwwnk
STEP: creating a claim
Jun 14 16:27:28.202: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-gj6b
STEP: Creating a pod to test subpath
Jun 14 16:27:28.599: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-gj6b" in namespace "provisioning-555" to be "Succeeded or Failed"
Jun 14 16:27:28.727: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 128.487569ms
Jun 14 16:27:30.856: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 2.256867028s
Jun 14 16:27:32.985: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 4.385973085s
Jun 14 16:27:35.115: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 6.515850882s
Jun 14 16:27:37.243: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 8.644710157s
Jun 14 16:27:39.373: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 10.773832038s
Jun 14 16:27:41.502: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 12.903240322s
Jun 14 16:27:43.633: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 15.034348313s
Jun 14 16:27:45.762: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 17.163374444s
Jun 14 16:27:47.892: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 19.293176516s
Jun 14 16:27:50.022: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Pending", Reason="", readiness=false. Elapsed: 21.423183018s
Jun 14 16:27:52.152: INFO: Pod "pod-subpath-test-dynamicpv-gj6b": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.552846972s
STEP: Saw pod success
Jun 14 16:27:52.152: INFO: Pod "pod-subpath-test-dynamicpv-gj6b" satisfied condition "Succeeded or Failed"
Jun 14 16:27:52.281: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-gj6b container test-container-subpath-dynamicpv-gj6b: <nil>
STEP: delete the pod
Jun 14 16:27:52.558: INFO: Waiting for pod pod-subpath-test-dynamicpv-gj6b to disappear
Jun 14 16:27:52.687: INFO: Pod pod-subpath-test-dynamicpv-gj6b no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-gj6b
Jun 14 16:27:52.687: INFO: Deleting pod "pod-subpath-test-dynamicpv-gj6b" in namespace "provisioning-555"
... skipping 231 lines ...
Jun 14 16:25:02.260: INFO: Creating resource for dynamic PV
Jun 14 16:25:02.260: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass snapshotting-7156-e2e-sctv4ww
STEP: creating a claim
Jun 14 16:25:02.394: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: [init] starting a pod to use the claim
Jun 14 16:25:02.792: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-kp4s6" in namespace "snapshotting-7156" to be "Succeeded or Failed"
Jun 14 16:25:02.924: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 131.491275ms
Jun 14 16:25:05.056: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.263474973s
Jun 14 16:25:07.188: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.395533668s
Jun 14 16:25:09.321: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.52812345s
Jun 14 16:25:11.453: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.660749767s
Jun 14 16:25:13.587: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.794359553s
Jun 14 16:25:15.725: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 12.932429033s
Jun 14 16:25:17.855: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 15.062196435s
Jun 14 16:25:19.985: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 17.192548001s
Jun 14 16:25:22.114: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 19.321224725s
Jun 14 16:25:24.242: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Pending", Reason="", readiness=false. Elapsed: 21.449890624s
Jun 14 16:25:26.372: INFO: Pod "pvc-snapshottable-tester-kp4s6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.579458778s
STEP: Saw pod success
Jun 14 16:25:26.372: INFO: Pod "pvc-snapshottable-tester-kp4s6" satisfied condition "Succeeded or Failed"
Jun 14 16:25:26.631: INFO: Pod pvc-snapshottable-tester-kp4s6 has the following logs: 
Jun 14 16:25:26.631: INFO: Deleting pod "pvc-snapshottable-tester-kp4s6" in namespace "snapshotting-7156"
Jun 14 16:25:26.769: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-kp4s6" to be fully deleted
Jun 14 16:25:26.897: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [ebs.csi.aws.comjbddc] to have phase Bound
Jun 14 16:25:27.026: INFO: PersistentVolumeClaim ebs.csi.aws.comjbddc found and phase=Bound (128.411141ms)
STEP: [init] checking the claim
... skipping 44 lines ...
[It] should check snapshot fields, check restore correctly works after modifying source data, check deletion
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:243
STEP: checking the snapshot
STEP: checking the SnapshotContent
STEP: Modifying source data test
STEP: modifying the data in the source PVC
Jun 14 16:26:03.885: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-x7cn6" in namespace "snapshotting-7156" to be "Succeeded or Failed"
Jun 14 16:26:04.014: INFO: Pod "pvc-snapshottable-data-tester-x7cn6": Phase="Pending", Reason="", readiness=false. Elapsed: 128.626873ms
Jun 14 16:26:06.142: INFO: Pod "pvc-snapshottable-data-tester-x7cn6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.25731732s
Jun 14 16:26:08.272: INFO: Pod "pvc-snapshottable-data-tester-x7cn6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.38713967s
Jun 14 16:26:10.405: INFO: Pod "pvc-snapshottable-data-tester-x7cn6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.519578371s
Jun 14 16:26:12.536: INFO: Pod "pvc-snapshottable-data-tester-x7cn6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.651531912s
Jun 14 16:26:14.665: INFO: Pod "pvc-snapshottable-data-tester-x7cn6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.7805184s
STEP: Saw pod success
Jun 14 16:26:14.666: INFO: Pod "pvc-snapshottable-data-tester-x7cn6" satisfied condition "Succeeded or Failed"
Jun 14 16:26:14.925: INFO: Pod pvc-snapshottable-data-tester-x7cn6 has the following logs: 
Jun 14 16:26:14.925: INFO: Deleting pod "pvc-snapshottable-data-tester-x7cn6" in namespace "snapshotting-7156"
Jun 14 16:26:15.066: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-x7cn6" to be fully deleted
STEP: creating a pvc from the snapshot
STEP: starting a pod to use the claim
Jun 14 16:26:39.730: INFO: Running '/usr/local/bin/kubectl --server=https://api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=snapshotting-7156 exec restored-pvc-tester-sjvxd --namespace=snapshotting-7156 -- cat /mnt/test/data'
... skipping 30 lines ...
Jun 14 16:27:05.420: INFO: volumesnapshotcontents pre-provisioned-snapcontent-06e3cb0b-e7de-4467-a753-bef66ed66708 has been found and is not deleted
Jun 14 16:27:06.550: INFO: volumesnapshotcontents pre-provisioned-snapcontent-06e3cb0b-e7de-4467-a753-bef66ed66708 has been found and is not deleted
Jun 14 16:27:07.680: INFO: volumesnapshotcontents pre-provisioned-snapcontent-06e3cb0b-e7de-4467-a753-bef66ed66708 has been found and is not deleted
Jun 14 16:27:08.809: INFO: volumesnapshotcontents pre-provisioned-snapcontent-06e3cb0b-e7de-4467-a753-bef66ed66708 has been found and is not deleted
Jun 14 16:27:09.939: INFO: volumesnapshotcontents pre-provisioned-snapcontent-06e3cb0b-e7de-4467-a753-bef66ed66708 has been found and is not deleted
Jun 14 16:27:11.068: INFO: volumesnapshotcontents pre-provisioned-snapcontent-06e3cb0b-e7de-4467-a753-bef66ed66708 has been found and is not deleted
Jun 14 16:27:12.068: INFO: WaitUntil failed after reaching the timeout 30s
[AfterEach] volume snapshot controller
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:221
Jun 14 16:27:12.198: INFO: Pod restored-pvc-tester-sjvxd has the following logs: 
Jun 14 16:27:12.198: INFO: Deleting pod "restored-pvc-tester-sjvxd" in namespace "snapshotting-7156"
Jun 14 16:27:12.329: INFO: Wait up to 5m0s for pod "restored-pvc-tester-sjvxd" to be fully deleted
Jun 14 16:28:18.590: INFO: deleting claim "snapshotting-7156"/"pvc-9n976"
... skipping 46 lines ...
Jun 14 16:27:48.901: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-19516xkn9
STEP: creating a claim
Jun 14 16:27:49.029: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-6gzv
STEP: Creating a pod to test subpath
Jun 14 16:27:49.423: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-6gzv" in namespace "provisioning-1951" to be "Succeeded or Failed"
Jun 14 16:27:49.551: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 128.252762ms
Jun 14 16:27:51.680: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.256763965s
Jun 14 16:27:53.809: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.385587909s
Jun 14 16:27:55.941: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.517519007s
Jun 14 16:27:58.070: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.647069711s
Jun 14 16:28:00.200: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.776547303s
Jun 14 16:28:02.329: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 12.905748219s
Jun 14 16:28:04.457: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Pending", Reason="", readiness=false. Elapsed: 15.034325752s
Jun 14 16:28:06.587: INFO: Pod "pod-subpath-test-dynamicpv-6gzv": Phase="Succeeded", Reason="", readiness=false. Elapsed: 17.163862125s
STEP: Saw pod success
Jun 14 16:28:06.587: INFO: Pod "pod-subpath-test-dynamicpv-6gzv" satisfied condition "Succeeded or Failed"
Jun 14 16:28:06.715: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-6gzv container test-container-subpath-dynamicpv-6gzv: <nil>
STEP: delete the pod
Jun 14 16:28:06.987: INFO: Waiting for pod pod-subpath-test-dynamicpv-6gzv to disappear
Jun 14 16:28:07.115: INFO: Pod pod-subpath-test-dynamicpv-6gzv no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-6gzv
Jun 14 16:28:07.116: INFO: Deleting pod "pod-subpath-test-dynamicpv-6gzv" in namespace "provisioning-1951"
... skipping 35 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Inline-volume (default fs)] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278

      Driver supports dynamic provisioning, skipping InlineVolume pattern

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:233
------------------------------
... skipping 348 lines ...
Jun 14 16:27:23.470: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-2556bp78l
STEP: creating a claim
Jun 14 16:27:23.605: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-jnj2
STEP: Creating a pod to test atomic-volume-subpath
Jun 14 16:27:24.010: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-jnj2" in namespace "provisioning-2556" to be "Succeeded or Failed"
Jun 14 16:27:24.142: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Pending", Reason="", readiness=false. Elapsed: 132.151846ms
Jun 14 16:27:26.276: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.266644904s
Jun 14 16:27:28.410: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.399901357s
Jun 14 16:27:30.542: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.532565341s
Jun 14 16:27:32.679: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.669032957s
Jun 14 16:27:34.812: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.80270303s
... skipping 18 lines ...
Jun 14 16:28:15.365: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Running", Reason="", readiness=true. Elapsed: 51.354961554s
Jun 14 16:28:17.498: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Running", Reason="", readiness=true. Elapsed: 53.488661788s
Jun 14 16:28:19.632: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Running", Reason="", readiness=true. Elapsed: 55.622632213s
Jun 14 16:28:21.765: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Running", Reason="", readiness=true. Elapsed: 57.755019016s
Jun 14 16:28:23.898: INFO: Pod "pod-subpath-test-dynamicpv-jnj2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 59.887968991s
STEP: Saw pod success
Jun 14 16:28:23.898: INFO: Pod "pod-subpath-test-dynamicpv-jnj2" satisfied condition "Succeeded or Failed"
Jun 14 16:28:24.033: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-jnj2 container test-container-subpath-dynamicpv-jnj2: <nil>
STEP: delete the pod
Jun 14 16:28:24.311: INFO: Waiting for pod pod-subpath-test-dynamicpv-jnj2 to disappear
Jun 14 16:28:24.443: INFO: Pod pod-subpath-test-dynamicpv-jnj2 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-jnj2
Jun 14 16:28:24.443: INFO: Deleting pod "pod-subpath-test-dynamicpv-jnj2" in namespace "provisioning-2556"
... skipping 138 lines ...
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:27:19.769: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if subpath file is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256
Jun 14 16:27:20.421: INFO: Creating resource for dynamic PV
Jun 14 16:27:20.421: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-8125-e2e-scq585q
STEP: creating a claim
Jun 14 16:27:20.552: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-n25m
STEP: Checking for subpath error in container status
Jun 14 16:28:33.211: INFO: Deleting pod "pod-subpath-test-dynamicpv-n25m" in namespace "provisioning-8125"
Jun 14 16:28:33.345: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-n25m" to be fully deleted
STEP: Deleting pod
Jun 14 16:28:41.607: INFO: Deleting pod "pod-subpath-test-dynamicpv-n25m" in namespace "provisioning-8125"
STEP: Deleting pvc
Jun 14 16:28:41.997: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com2p7pr"
... skipping 10 lines ...

• [SLOW TEST:93.160 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath file is outside the volume [Slow][LinuxOnly]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256
------------------------------
[BeforeEach] [sig-storage] Storage Policy Based Volume Provisioning [Feature:vsphere]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:28:52.778: INFO: >>> kubeConfig: /root/.kube/config
... skipping 351 lines ...
Jun 14 16:27:45.714: INFO: Creating resource for dynamic PV
Jun 14 16:27:45.714: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-21198hstx
STEP: creating a claim
Jun 14 16:27:45.843: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod to format volume volume-prep-provisioning-2119
Jun 14 16:27:46.230: INFO: Waiting up to 5m0s for pod "volume-prep-provisioning-2119" in namespace "provisioning-2119" to be "Succeeded or Failed"
Jun 14 16:27:46.363: INFO: Pod "volume-prep-provisioning-2119": Phase="Pending", Reason="", readiness=false. Elapsed: 132.242102ms
Jun 14 16:27:48.494: INFO: Pod "volume-prep-provisioning-2119": Phase="Pending", Reason="", readiness=false. Elapsed: 2.263480721s
Jun 14 16:27:50.622: INFO: Pod "volume-prep-provisioning-2119": Phase="Pending", Reason="", readiness=false. Elapsed: 4.391929246s
Jun 14 16:27:52.754: INFO: Pod "volume-prep-provisioning-2119": Phase="Pending", Reason="", readiness=false. Elapsed: 6.523710172s
Jun 14 16:27:54.884: INFO: Pod "volume-prep-provisioning-2119": Phase="Pending", Reason="", readiness=false. Elapsed: 8.653740963s
Jun 14 16:27:57.013: INFO: Pod "volume-prep-provisioning-2119": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.782464648s
STEP: Saw pod success
Jun 14 16:27:57.013: INFO: Pod "volume-prep-provisioning-2119" satisfied condition "Succeeded or Failed"
Jun 14 16:27:57.013: INFO: Deleting pod "volume-prep-provisioning-2119" in namespace "provisioning-2119"
Jun 14 16:27:57.148: INFO: Wait up to 5m0s for pod "volume-prep-provisioning-2119" to be fully deleted
STEP: Creating pod pod-subpath-test-dynamicpv-pzxs
STEP: Checking for subpath error in container status
Jun 14 16:28:31.672: INFO: Deleting pod "pod-subpath-test-dynamicpv-pzxs" in namespace "provisioning-2119"
Jun 14 16:28:31.805: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-pzxs" to be fully deleted
STEP: Deleting pod
Jun 14 16:28:31.934: INFO: Deleting pod "pod-subpath-test-dynamicpv-pzxs" in namespace "provisioning-2119"
STEP: Deleting pvc
Jun 14 16:28:32.319: INFO: Deleting PersistentVolumeClaim "aws2v546"
... skipping 286 lines ...
Jun 14 16:28:47.939: INFO: Waiting for pod aws-client to disappear
Jun 14 16:28:48.071: INFO: Pod aws-client no longer exists
STEP: cleaning the environment after aws
STEP: Deleting pv and pvc
Jun 14 16:28:48.071: INFO: Deleting PersistentVolumeClaim "pvc-dk6lg"
Jun 14 16:28:48.201: INFO: Deleting PersistentVolume "aws-2kxkm"
Jun 14 16:28:49.096: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-091d9b372c0ab22b8", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-091d9b372c0ab22b8 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 4eeb4cf4-20ba-40b4-9478-7e4d5b60fd17
Jun 14 16:28:54.909: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-091d9b372c0ab22b8", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-091d9b372c0ab22b8 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: fb886d34-b37b-40d1-ac2b-96ebf6c93c45
Jun 14 16:29:00.671: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-091d9b372c0ab22b8", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-091d9b372c0ab22b8 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 17bc5d61-fd2d-4997-900c-2b081f2053ee
Jun 14 16:29:06.442: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-091d9b372c0ab22b8".
[AfterEach] [Testpattern: Pre-provisioned PV (default fs)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:29:06.442: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-5123" for this suite.
... skipping 150 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267

      Distro debian doesn't support ntfs -- skipping

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:127
------------------------------
... skipping 6 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:267

    Driver ebs.csi.aws.com doesn't support ntfs -- skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:121
------------------------------
... skipping 98 lines ...
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:28:54.342: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename volumemode
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to use a volume in a pod with mismatched mode [Slow]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
Jun 14 16:28:54.970: INFO: Creating resource for dynamic PV
Jun 14 16:28:54.970: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass volumemode-7576-e2e-sc7nlhz
STEP: creating a claim
STEP: Creating pod
STEP: Waiting for the pod to fail
Jun 14 16:28:59.862: INFO: Deleting pod "pod-d83ff951-394a-4c27-a5ec-a9d3f01b3d7e" in namespace "volumemode-7576"
Jun 14 16:28:59.996: INFO: Wait up to 5m0s for pod "pod-d83ff951-394a-4c27-a5ec-a9d3f01b3d7e" to be fully deleted
STEP: Deleting pvc
Jun 14 16:29:08.506: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comd9kpg"
Jun 14 16:29:08.633: INFO: Waiting up to 5m0s for PersistentVolume pvc-3ac19ef5-4758-419e-bd61-9d38d2559aaa to get deleted
Jun 14 16:29:08.759: INFO: PersistentVolume pvc-3ac19ef5-4758-419e-bd61-9d38d2559aaa found and phase=Released (126.024997ms)
... skipping 9 lines ...

• [SLOW TEST:30.184 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (block volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to use a volume in a pod with mismatched mode [Slow]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
... skipping 6 lines ...
Jun 14 16:28:04.592: INFO: Creating resource for dynamic PV
Jun 14 16:28:04.592: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-6012-e2e-scw95wh
STEP: creating a claim
Jun 14 16:28:04.720: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod to format volume volume-prep-provisioning-6012
Jun 14 16:28:05.106: INFO: Waiting up to 5m0s for pod "volume-prep-provisioning-6012" in namespace "provisioning-6012" to be "Succeeded or Failed"
Jun 14 16:28:05.233: INFO: Pod "volume-prep-provisioning-6012": Phase="Pending", Reason="", readiness=false. Elapsed: 126.803123ms
Jun 14 16:28:07.361: INFO: Pod "volume-prep-provisioning-6012": Phase="Pending", Reason="", readiness=false. Elapsed: 2.254402013s
Jun 14 16:28:09.490: INFO: Pod "volume-prep-provisioning-6012": Phase="Pending", Reason="", readiness=false. Elapsed: 4.384019757s
Jun 14 16:28:11.618: INFO: Pod "volume-prep-provisioning-6012": Phase="Pending", Reason="", readiness=false. Elapsed: 6.511241407s
Jun 14 16:28:13.745: INFO: Pod "volume-prep-provisioning-6012": Phase="Pending", Reason="", readiness=false. Elapsed: 8.63850029s
Jun 14 16:28:15.880: INFO: Pod "volume-prep-provisioning-6012": Phase="Pending", Reason="", readiness=false. Elapsed: 10.773319665s
Jun 14 16:28:18.007: INFO: Pod "volume-prep-provisioning-6012": Phase="Succeeded", Reason="", readiness=false. Elapsed: 12.900454917s
STEP: Saw pod success
Jun 14 16:28:18.007: INFO: Pod "volume-prep-provisioning-6012" satisfied condition "Succeeded or Failed"
Jun 14 16:28:18.007: INFO: Deleting pod "volume-prep-provisioning-6012" in namespace "provisioning-6012"
Jun 14 16:28:18.140: INFO: Wait up to 5m0s for pod "volume-prep-provisioning-6012" to be fully deleted
STEP: Creating pod pod-subpath-test-dynamicpv-7xw6
STEP: Checking for subpath error in container status
Jun 14 16:28:54.655: INFO: Deleting pod "pod-subpath-test-dynamicpv-7xw6" in namespace "provisioning-6012"
Jun 14 16:28:54.790: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-7xw6" to be fully deleted
STEP: Deleting pod
Jun 14 16:28:54.917: INFO: Deleting pod "pod-subpath-test-dynamicpv-7xw6" in namespace "provisioning-6012"
STEP: Deleting pvc
Jun 14 16:28:55.297: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com7ddz9"
... skipping 74 lines ...
[BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:28:59.249: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename volumemode
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to use a volume in a pod with mismatched mode [Slow]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
Jun 14 16:28:59.892: INFO: Creating resource for dynamic PV
Jun 14 16:28:59.892: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass volumemode-6815-e2e-sc7tfwg
STEP: creating a claim
STEP: Creating pod
STEP: Waiting for the pod to fail
Jun 14 16:29:06.808: INFO: Deleting pod "pod-6a72dafa-2d9e-4a00-942a-c6294e953215" in namespace "volumemode-6815"
Jun 14 16:29:06.937: INFO: Wait up to 5m0s for pod "pod-6a72dafa-2d9e-4a00-942a-c6294e953215" to be fully deleted
STEP: Deleting pvc
Jun 14 16:29:17.451: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.comt4tkh"
Jun 14 16:29:17.580: INFO: Waiting up to 5m0s for PersistentVolume pvc-e2eadc00-e44d-418b-92c1-36bc5d9f8274 to get deleted
Jun 14 16:29:17.708: INFO: PersistentVolume pvc-e2eadc00-e44d-418b-92c1-36bc5d9f8274 found and phase=Released (127.203253ms)
... skipping 9 lines ...

• [SLOW TEST:34.258 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to use a volume in a pod with mismatched mode [Slow]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
... skipping 273 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to use a volume in a pod with mismatched mode [Slow] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:296

    Driver "ebs.csi.aws.com" does not support volume type "PreprovisionedPV" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 130 lines ...
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:28:52.932: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename provisioning
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278
Jun 14 16:28:53.584: INFO: Creating resource for dynamic PV
Jun 14 16:28:53.584: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-8424-e2e-sczm46b
STEP: creating a claim
Jun 14 16:28:53.716: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-snqp
STEP: Checking for subpath error in container status
Jun 14 16:29:16.382: INFO: Deleting pod "pod-subpath-test-dynamicpv-snqp" in namespace "provisioning-8424"
Jun 14 16:29:16.513: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-snqp" to be fully deleted
STEP: Deleting pod
Jun 14 16:29:26.775: INFO: Deleting pod "pod-subpath-test-dynamicpv-snqp" in namespace "provisioning-8424"
STEP: Deleting pvc
Jun 14 16:29:27.166: INFO: Deleting PersistentVolumeClaim "ebs.csi.aws.com2jw85"
... skipping 11 lines ...

• [SLOW TEST:50.287 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:278
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
... skipping 130 lines ...
Jun 14 16:29:34.369: INFO: Pod aws-client still exists
Jun 14 16:29:36.233: INFO: Waiting for pod aws-client to disappear
Jun 14 16:29:36.369: INFO: Pod aws-client still exists
Jun 14 16:29:38.233: INFO: Waiting for pod aws-client to disappear
Jun 14 16:29:38.375: INFO: Pod aws-client no longer exists
STEP: cleaning the environment after aws
Jun 14 16:29:39.053: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-0a36101710cb51968", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-0a36101710cb51968 is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 22819930-533d-4c22-ac9f-2bc8f43a88db
Jun 14 16:29:44.823: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-0a36101710cb51968".
[AfterEach] [Testpattern: Inline-volume (ext4)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:29:44.823: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-7533" for this suite.
... skipping 150 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240

    Driver ebs.csi.aws.com doesn't support ntfs -- skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:121
------------------------------
... skipping 6 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:256

    Driver ebs.csi.aws.com doesn't support ntfs -- skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:121
------------------------------
... skipping 12 lines ...
Jun 14 16:29:07.358: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass volume-9718jg2s4
STEP: creating a claim
Jun 14 16:29:07.490: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod exec-volume-test-dynamicpv-khxt
STEP: Creating a pod to test exec-volume-test
Jun 14 16:29:07.882: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-khxt" in namespace "volume-9718" to be "Succeeded or Failed"
Jun 14 16:29:08.010: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 128.566194ms
Jun 14 16:29:10.139: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.257615272s
Jun 14 16:29:12.269: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 4.387038754s
Jun 14 16:29:14.398: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 6.516227911s
Jun 14 16:29:16.527: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 8.645003623s
Jun 14 16:29:18.660: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 10.778087652s
... skipping 8 lines ...
Jun 14 16:29:37.823: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 29.941745809s
Jun 14 16:29:39.953: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 32.071660405s
Jun 14 16:29:42.083: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 34.201136349s
Jun 14 16:29:44.212: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Pending", Reason="", readiness=false. Elapsed: 36.330269148s
Jun 14 16:29:46.341: INFO: Pod "exec-volume-test-dynamicpv-khxt": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.459179622s
STEP: Saw pod success
Jun 14 16:29:46.341: INFO: Pod "exec-volume-test-dynamicpv-khxt" satisfied condition "Succeeded or Failed"
Jun 14 16:29:46.511: INFO: Trying to get logs from node ip-172-20-56-37.ap-northeast-1.compute.internal pod exec-volume-test-dynamicpv-khxt container exec-container-dynamicpv-khxt: <nil>
STEP: delete the pod
Jun 14 16:29:46.784: INFO: Waiting for pod exec-volume-test-dynamicpv-khxt to disappear
Jun 14 16:29:46.914: INFO: Pod exec-volume-test-dynamicpv-khxt no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-khxt
Jun 14 16:29:46.914: INFO: Deleting pod "exec-volume-test-dynamicpv-khxt" in namespace "volume-9718"
... skipping 184 lines ...
Jun 14 16:27:54.620: INFO: Creating resource for dynamic PV
Jun 14 16:27:54.620: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass snapshotting-8482-e2e-scs7b7j
STEP: creating a claim
Jun 14 16:27:54.748: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: [init] starting a pod to use the claim
Jun 14 16:27:55.129: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-5z45d" in namespace "snapshotting-8482" to be "Succeeded or Failed"
Jun 14 16:27:55.255: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 126.124255ms
Jun 14 16:27:57.383: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 2.253528857s
Jun 14 16:27:59.509: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 4.379884781s
Jun 14 16:28:01.637: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 6.507390688s
Jun 14 16:28:03.764: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 8.63466713s
Jun 14 16:28:05.891: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 10.762070257s
Jun 14 16:28:08.019: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 12.889342123s
Jun 14 16:28:10.156: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 15.026580247s
Jun 14 16:28:12.283: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 17.154204646s
Jun 14 16:28:14.411: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Pending", Reason="", readiness=false. Elapsed: 19.281240102s
Jun 14 16:28:16.537: INFO: Pod "pvc-snapshottable-tester-5z45d": Phase="Succeeded", Reason="", readiness=false. Elapsed: 21.407687643s
STEP: Saw pod success
Jun 14 16:28:16.537: INFO: Pod "pvc-snapshottable-tester-5z45d" satisfied condition "Succeeded or Failed"
Jun 14 16:28:16.842: INFO: Pod pvc-snapshottable-tester-5z45d has the following logs: 
Jun 14 16:28:16.842: INFO: Deleting pod "pvc-snapshottable-tester-5z45d" in namespace "snapshotting-8482"
Jun 14 16:28:16.986: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-5z45d" to be fully deleted
Jun 14 16:28:17.113: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [ebs.csi.aws.commhfq7] to have phase Bound
Jun 14 16:28:17.240: INFO: PersistentVolumeClaim ebs.csi.aws.commhfq7 found and phase=Bound (127.198448ms)
STEP: [init] checking the claim
... skipping 14 lines ...
[It] should check snapshot fields, check restore correctly works after modifying source data, check deletion
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:243
STEP: checking the snapshot
STEP: checking the SnapshotContent
STEP: Modifying source data test
STEP: modifying the data in the source PVC
Jun 14 16:28:27.155: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-p69q5" in namespace "snapshotting-8482" to be "Succeeded or Failed"
Jun 14 16:28:27.281: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 126.04396ms
Jun 14 16:28:29.409: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.253592803s
Jun 14 16:28:31.536: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.38025395s
Jun 14 16:28:33.662: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.50695772s
Jun 14 16:28:35.791: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.635638599s
Jun 14 16:28:37.927: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.771998681s
... skipping 6 lines ...
Jun 14 16:28:52.825: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 25.669953887s
Jun 14 16:28:54.953: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 27.797141132s
Jun 14 16:28:57.080: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 29.924284187s
Jun 14 16:28:59.207: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Pending", Reason="", readiness=false. Elapsed: 32.051125891s
Jun 14 16:29:01.334: INFO: Pod "pvc-snapshottable-data-tester-p69q5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 34.178327173s
STEP: Saw pod success
Jun 14 16:29:01.334: INFO: Pod "pvc-snapshottable-data-tester-p69q5" satisfied condition "Succeeded or Failed"
Jun 14 16:29:01.589: INFO: Pod pvc-snapshottable-data-tester-p69q5 has the following logs: 
Jun 14 16:29:01.589: INFO: Deleting pod "pvc-snapshottable-data-tester-p69q5" in namespace "snapshotting-8482"
Jun 14 16:29:01.719: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-p69q5" to be fully deleted
STEP: creating a pvc from the snapshot
STEP: starting a pod to use the claim
Jun 14 16:29:16.358: INFO: Running '/usr/local/bin/kubectl --server=https://api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=snapshotting-8482 exec restored-pvc-tester-pn7ld --namespace=snapshotting-8482 -- cat /mnt/test/data'
... skipping 378 lines ...
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:185
STEP: Creating a kubernetes client
Jun 14 16:30:18.947: INFO: >>> kubeConfig: /root/.kube/config
STEP: Building a namespace api object, basename topology
STEP: Waiting for a default service account to be provisioned in namespace
[It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192
Jun 14 16:30:19.724: INFO: found topology map[topology.kubernetes.io/zone:ap-northeast-1a]
Jun 14 16:30:19.724: INFO: In-tree plugin kubernetes.io/aws-ebs is not migrated, not validating any metrics
Jun 14 16:30:19.724: INFO: Not enough topologies in cluster -- skipping
STEP: Deleting pvc
STEP: Deleting sc
... skipping 7 lines ...
[ebs-csi-migration] EBS CSI Migration
/tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:85
  [Driver: aws]
  /tmp/kops.Ngif65n0h/tests/e2e-kubernetes/e2e_test.go:91
    [Testpattern: Dynamic PV (delayed binding)] topology
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
      should fail to schedule a pod which has topologies that conflict with AllowedTopologies [It]
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

      Not enough topologies in cluster -- skipping

      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:199
------------------------------
... skipping 189 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: ebs.csi.aws.com]
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:175
  [Testpattern: Pre-provisioned PV (default fs)] subPath
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach]
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:240

    Driver "ebs.csi.aws.com" does not support volume type "PreprovisionedPV" - skipping

    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/external/external.go:255
------------------------------
... skipping 32 lines ...
Jun 14 16:29:45.785: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-2710-e2e-schlphm
STEP: creating a claim
Jun 14 16:29:45.921: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-p2p6
STEP: Creating a pod to test subpath
Jun 14 16:29:46.330: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-p2p6" in namespace "provisioning-2710" to be "Succeeded or Failed"
Jun 14 16:29:46.513: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 182.888218ms
Jun 14 16:29:48.650: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.319121234s
Jun 14 16:29:50.786: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.455571347s
Jun 14 16:29:52.923: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.59287008s
Jun 14 16:29:55.061: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.730816193s
Jun 14 16:29:57.198: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.867834544s
Jun 14 16:29:59.336: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 13.005134949s
Jun 14 16:30:01.472: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 15.141890925s
Jun 14 16:30:03.609: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 17.278442342s
Jun 14 16:30:05.746: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 19.415221577s
Jun 14 16:30:07.883: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Pending", Reason="", readiness=false. Elapsed: 21.552461841s
Jun 14 16:30:10.019: INFO: Pod "pod-subpath-test-dynamicpv-p2p6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.688794186s
STEP: Saw pod success
Jun 14 16:30:10.019: INFO: Pod "pod-subpath-test-dynamicpv-p2p6" satisfied condition "Succeeded or Failed"
Jun 14 16:30:10.158: INFO: Trying to get logs from node ip-172-20-56-37.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-p2p6 container test-container-volume-dynamicpv-p2p6: <nil>
STEP: delete the pod
Jun 14 16:30:10.460: INFO: Waiting for pod pod-subpath-test-dynamicpv-p2p6 to disappear
Jun 14 16:30:10.596: INFO: Pod pod-subpath-test-dynamicpv-p2p6 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-p2p6
Jun 14 16:30:10.596: INFO: Deleting pod "pod-subpath-test-dynamicpv-p2p6" in namespace "provisioning-2710"
... skipping 121 lines ...
Jun 14 16:29:34.151: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(aws) supported size:{ 1Mi} 
STEP: creating a StorageClass volume-5968msmz9
STEP: creating a claim
Jun 14 16:29:34.279: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod exec-volume-test-dynamicpv-gs2q
STEP: Creating a pod to test exec-volume-test
Jun 14 16:29:34.670: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-gs2q" in namespace "volume-5968" to be "Succeeded or Failed"
Jun 14 16:29:34.797: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 127.555869ms
Jun 14 16:29:36.932: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 2.262539645s
Jun 14 16:29:39.061: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 4.391298997s
Jun 14 16:29:41.201: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 6.53110325s
Jun 14 16:29:43.330: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 8.660184314s
Jun 14 16:29:45.458: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 10.788003699s
... skipping 8 lines ...
Jun 14 16:30:04.612: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 29.942225753s
Jun 14 16:30:06.740: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 32.07062318s
Jun 14 16:30:08.869: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 34.198728851s
Jun 14 16:30:10.997: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Pending", Reason="", readiness=false. Elapsed: 36.326975018s
Jun 14 16:30:13.125: INFO: Pod "exec-volume-test-dynamicpv-gs2q": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.455061451s
STEP: Saw pod success
Jun 14 16:30:13.125: INFO: Pod "exec-volume-test-dynamicpv-gs2q" satisfied condition "Succeeded or Failed"
Jun 14 16:30:13.252: INFO: Trying to get logs from node ip-172-20-51-223.ap-northeast-1.compute.internal pod exec-volume-test-dynamicpv-gs2q container exec-container-dynamicpv-gs2q: <nil>
STEP: delete the pod
Jun 14 16:30:13.523: INFO: Waiting for pod exec-volume-test-dynamicpv-gs2q to disappear
Jun 14 16:30:13.651: INFO: Pod exec-volume-test-dynamicpv-gs2q no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-gs2q
Jun 14 16:30:13.651: INFO: Deleting pod "exec-volume-test-dynamicpv-gs2q" in namespace "volume-5968"
... skipping 35 lines ...
Jun 14 16:30:05.094: INFO: Using claimSize:1Gi, test suite supported size:{ 1Gi}, driver(ebs.csi.aws.com) supported size:{ 1Gi} 
STEP: creating a StorageClass volume-expand-8507-e2e-sctgd75
STEP: creating a claim
Jun 14 16:30:05.231: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Expanding non-expandable pvc
Jun 14 16:30:05.507: INFO: currentPvcSize {{1073741824 0} {<nil>} 1Gi BinarySI}, newSize {{2147483648 0} {<nil>}  BinarySI}
Jun 14 16:30:05.781: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:08.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:10.056: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:12.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:14.056: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:16.056: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:18.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:20.057: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:22.056: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:24.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:26.056: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:28.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:30.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:32.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:34.055: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:36.056: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 5 lines ...
  	},
  	VolumeName:       "",
  	StorageClassName: &"volume-expand-8507-e2e-sctgd75",
  	... // 2 identical fields
  }

Jun 14 16:30:36.330: INFO: Error updating pvc ebs.csi.aws.combpjgg: PersistentVolumeClaim "ebs.csi.aws.combpjgg" is invalid: spec: Forbidden: spec is immutable after creation except resources.requests for bound claims
  core.PersistentVolumeClaimSpec{
  	AccessModes: {"ReadWriteOnce"},
  	Selector:    nil,
  	Resources: core.ResourceRequirements{
  		Limits: nil,
- 		Requests: core.ResourceList{
... skipping 111 lines ...
Jun 14 16:30:30.670: INFO: Waiting for pod aws-client to disappear
Jun 14 16:30:30.796: INFO: Pod aws-client no longer exists
STEP: cleaning the environment after aws
STEP: Deleting pv and pvc
Jun 14 16:30:30.796: INFO: Deleting PersistentVolumeClaim "pvc-26825"
Jun 14 16:30:30.925: INFO: Deleting PersistentVolume "aws-98pkj"
Jun 14 16:30:31.836: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-09d9c868f1d418ebc", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-09d9c868f1d418ebc is currently attached to i-0fd2e5eaedfcde5eb
	status code: 400, request id: ebe92141-26fe-4f4a-baef-297bb9707310
Jun 14 16:30:37.656: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-09d9c868f1d418ebc", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-09d9c868f1d418ebc is currently attached to i-0fd2e5eaedfcde5eb
	status code: 400, request id: 4a14d139-f165-498b-bf69-0358d9afeb19
Jun 14 16:30:43.412: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-09d9c868f1d418ebc".
[AfterEach] [Testpattern: Pre-provisioned PV (xfs)][Slow] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:30:43.412: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-6469" for this suite.
... skipping 166 lines ...
Jun 14 16:30:00.918: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass volume-6527-e2e-scc6jwr
STEP: creating a claim
Jun 14 16:30:01.049: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod exec-volume-test-dynamicpv-rrtz
STEP: Creating a pod to test exec-volume-test
Jun 14 16:30:01.448: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-rrtz" in namespace "volume-6527" to be "Succeeded or Failed"
Jun 14 16:30:01.578: INFO: Pod "exec-volume-test-dynamicpv-rrtz": Phase="Pending", Reason="", readiness=false. Elapsed: 129.938202ms
Jun 14 16:30:03.708: INFO: Pod "exec-volume-test-dynamicpv-rrtz": Phase="Pending", Reason="", readiness=false. Elapsed: 2.260058647s
Jun 14 16:30:05.838: INFO: Pod "exec-volume-test-dynamicpv-rrtz": Phase="Pending", Reason="", readiness=false. Elapsed: 4.390024694s
Jun 14 16:30:07.969: INFO: Pod "exec-volume-test-dynamicpv-rrtz": Phase="Pending", Reason="", readiness=false. Elapsed: 6.521056457s
Jun 14 16:30:10.102: INFO: Pod "exec-volume-test-dynamicpv-rrtz": Phase="Pending", Reason="", readiness=false. Elapsed: 8.65395233s
Jun 14 16:30:12.234: INFO: Pod "exec-volume-test-dynamicpv-rrtz": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.785296063s
STEP: Saw pod success
Jun 14 16:30:12.234: INFO: Pod "exec-volume-test-dynamicpv-rrtz" satisfied condition "Succeeded or Failed"
Jun 14 16:30:12.366: INFO: Trying to get logs from node ip-172-20-51-223.ap-northeast-1.compute.internal pod exec-volume-test-dynamicpv-rrtz container exec-container-dynamicpv-rrtz: <nil>
STEP: delete the pod
Jun 14 16:30:12.635: INFO: Waiting for pod exec-volume-test-dynamicpv-rrtz to disappear
Jun 14 16:30:12.764: INFO: Pod exec-volume-test-dynamicpv-rrtz no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-rrtz
Jun 14 16:30:12.765: INFO: Deleting pod "exec-volume-test-dynamicpv-rrtz" in namespace "volume-6527"
... skipping 374 lines ...
Jun 14 16:30:47.189: INFO: Waiting for pod aws-client to disappear
Jun 14 16:30:47.316: INFO: Pod aws-client no longer exists
STEP: cleaning the environment after aws
STEP: Deleting pv and pvc
Jun 14 16:30:47.316: INFO: Deleting PersistentVolumeClaim "pvc-85fmm"
Jun 14 16:30:47.448: INFO: Deleting PersistentVolume "aws-9qgjj"
Jun 14 16:30:48.460: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-06e66bce00fa9499b", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-06e66bce00fa9499b is currently attached to i-07389f3062fba0924
	status code: 400, request id: c9a5d546-f083-44eb-ae67-b3637aa7f5e2
Jun 14 16:30:54.339: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-06e66bce00fa9499b", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-06e66bce00fa9499b is currently attached to i-07389f3062fba0924
	status code: 400, request id: c84421ff-f9c3-4f98-877c-62de9da06bda
Jun 14 16:31:00.173: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-06e66bce00fa9499b", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-06e66bce00fa9499b is currently attached to i-07389f3062fba0924
	status code: 400, request id: fc1c9a23-aafc-4d48-9aaf-a900849998f3
Jun 14 16:31:06.038: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-06e66bce00fa9499b".
[AfterEach] [Testpattern: Pre-provisioned PV (block volmode)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:31:06.038: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-6052" for this suite.
... skipping 419 lines ...
Jun 14 16:30:35.825: INFO: PersistentVolumeClaim pvc-9b884 found but phase is Pending instead of Bound.
Jun 14 16:30:37.954: INFO: PersistentVolumeClaim pvc-9b884 found and phase=Bound (15.03543898s)
Jun 14 16:30:37.954: INFO: Waiting up to 3m0s for PersistentVolume aws-kxqff to have phase Bound
Jun 14 16:30:38.083: INFO: PersistentVolume aws-kxqff found and phase=Bound (128.634298ms)
STEP: Creating pod exec-volume-test-preprovisionedpv-s7d9
STEP: Creating a pod to test exec-volume-test
Jun 14 16:30:38.506: INFO: Waiting up to 5m0s for pod "exec-volume-test-preprovisionedpv-s7d9" in namespace "volume-5263" to be "Succeeded or Failed"
Jun 14 16:30:38.635: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 128.715941ms
Jun 14 16:30:40.764: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.257478883s
Jun 14 16:30:42.893: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.38637481s
Jun 14 16:30:45.022: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.51586073s
Jun 14 16:30:47.151: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.644717278s
Jun 14 16:30:49.282: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.775782666s
... skipping 6 lines ...
Jun 14 16:31:04.190: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 25.6840832s
Jun 14 16:31:06.320: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 27.813642819s
Jun 14 16:31:08.462: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 29.955573167s
Jun 14 16:31:10.592: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Pending", Reason="", readiness=false. Elapsed: 32.085619802s
Jun 14 16:31:12.721: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 34.214680215s
STEP: Saw pod success
Jun 14 16:31:12.721: INFO: Pod "exec-volume-test-preprovisionedpv-s7d9" satisfied condition "Succeeded or Failed"
Jun 14 16:31:12.850: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod exec-volume-test-preprovisionedpv-s7d9 container exec-container-preprovisionedpv-s7d9: <nil>
STEP: delete the pod
Jun 14 16:31:13.117: INFO: Waiting for pod exec-volume-test-preprovisionedpv-s7d9 to disappear
Jun 14 16:31:13.246: INFO: Pod exec-volume-test-preprovisionedpv-s7d9 no longer exists
STEP: Deleting pod exec-volume-test-preprovisionedpv-s7d9
Jun 14 16:31:13.246: INFO: Deleting pod "exec-volume-test-preprovisionedpv-s7d9" in namespace "volume-5263"
STEP: Deleting pv and pvc
Jun 14 16:31:13.374: INFO: Deleting PersistentVolumeClaim "pvc-9b884"
Jun 14 16:31:13.505: INFO: Deleting PersistentVolume "aws-kxqff"
Jun 14 16:31:14.001: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-09c71958ea054385b", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-09c71958ea054385b is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 55c4f1d2-adf6-4fe3-b2a2-f3bb350b2e21
Jun 14 16:31:19.817: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-09c71958ea054385b", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-09c71958ea054385b is currently attached to i-01afea6f729a201fd
	status code: 400, request id: d2cbf605-bd96-4a73-b843-52a727f3ea57
Jun 14 16:31:25.541: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-09c71958ea054385b", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-09c71958ea054385b is currently attached to i-01afea6f729a201fd
	status code: 400, request id: 96277dca-951a-448d-bfeb-efb6f02fedce
Jun 14 16:31:31.289: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-09c71958ea054385b".
[AfterEach] [Testpattern: Pre-provisioned PV (xfs)][Slow] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:31:31.289: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-5263" for this suite.
... skipping 146 lines ...
Jun 14 16:30:26.352: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass provisioning-3290-e2e-sctjpqm
STEP: creating a claim
Jun 14 16:30:26.483: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: Creating pod pod-subpath-test-dynamicpv-cgdg
STEP: Creating a pod to test subpath
Jun 14 16:30:26.883: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-cgdg" in namespace "provisioning-3290" to be "Succeeded or Failed"
Jun 14 16:30:27.014: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 130.586936ms
Jun 14 16:30:29.145: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 2.262153164s
Jun 14 16:30:31.277: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 4.393894836s
Jun 14 16:30:33.409: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 6.525680038s
Jun 14 16:30:35.540: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 8.65677243s
Jun 14 16:30:37.671: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 10.787521642s
Jun 14 16:30:39.802: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 12.918521116s
Jun 14 16:30:41.932: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 15.049133213s
Jun 14 16:30:44.063: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 17.180327509s
Jun 14 16:30:46.195: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 19.312250495s
Jun 14 16:30:48.326: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 21.44340953s
Jun 14 16:30:50.463: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.580152273s
STEP: Saw pod success
Jun 14 16:30:50.463: INFO: Pod "pod-subpath-test-dynamicpv-cgdg" satisfied condition "Succeeded or Failed"
Jun 14 16:30:50.594: INFO: Trying to get logs from node ip-172-20-61-58.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-cgdg container test-container-subpath-dynamicpv-cgdg: <nil>
STEP: delete the pod
Jun 14 16:30:50.874: INFO: Waiting for pod pod-subpath-test-dynamicpv-cgdg to disappear
Jun 14 16:30:51.004: INFO: Pod pod-subpath-test-dynamicpv-cgdg no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-cgdg
Jun 14 16:30:51.005: INFO: Deleting pod "pod-subpath-test-dynamicpv-cgdg" in namespace "provisioning-3290"
STEP: Creating pod pod-subpath-test-dynamicpv-cgdg
STEP: Creating a pod to test subpath
Jun 14 16:30:51.267: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-cgdg" in namespace "provisioning-3290" to be "Succeeded or Failed"
Jun 14 16:30:51.398: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 130.904061ms
Jun 14 16:30:53.528: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 2.261543088s
Jun 14 16:30:55.666: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 4.399509084s
Jun 14 16:30:57.799: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 6.531686998s
Jun 14 16:30:59.934: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 8.66677644s
Jun 14 16:31:02.064: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 10.797213572s
... skipping 6 lines ...
Jun 14 16:31:16.989: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 25.721994828s
Jun 14 16:31:19.120: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 27.853194212s
Jun 14 16:31:21.252: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 29.985163953s
Jun 14 16:31:23.383: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Pending", Reason="", readiness=false. Elapsed: 32.116545126s
Jun 14 16:31:25.514: INFO: Pod "pod-subpath-test-dynamicpv-cgdg": Phase="Succeeded", Reason="", readiness=false. Elapsed: 34.247419393s
STEP: Saw pod success
Jun 14 16:31:25.514: INFO: Pod "pod-subpath-test-dynamicpv-cgdg" satisfied condition "Succeeded or Failed"
Jun 14 16:31:25.645: INFO: Trying to get logs from node ip-172-20-37-65.ap-northeast-1.compute.internal pod pod-subpath-test-dynamicpv-cgdg container test-container-subpath-dynamicpv-cgdg: <nil>
STEP: delete the pod
Jun 14 16:31:25.915: INFO: Waiting for pod pod-subpath-test-dynamicpv-cgdg to disappear
Jun 14 16:31:26.045: INFO: Pod pod-subpath-test-dynamicpv-cgdg no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-cgdg
Jun 14 16:31:26.045: INFO: Deleting pod "pod-subpath-test-dynamicpv-cgdg" in namespace "provisioning-3290"
... skipping 164 lines ...
Jun 14 16:29:54.525: INFO: Creating resource for dynamic PV
Jun 14 16:29:54.525: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass snapshotting-622-e2e-sc4smzv
STEP: creating a claim
Jun 14 16:29:54.654: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: [init] starting a pod to use the claim
Jun 14 16:29:55.033: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-24fk9" in namespace "snapshotting-622" to be "Succeeded or Failed"
Jun 14 16:29:55.160: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 126.530213ms
Jun 14 16:29:57.286: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.252753651s
Jun 14 16:29:59.412: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.378634255s
Jun 14 16:30:01.538: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.5048546s
Jun 14 16:30:03.664: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.630957514s
Jun 14 16:30:05.790: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.756837225s
Jun 14 16:30:07.915: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 12.88204159s
Jun 14 16:30:10.042: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 15.008661274s
Jun 14 16:30:12.168: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 17.134744462s
Jun 14 16:30:14.294: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Pending", Reason="", readiness=false. Elapsed: 19.26030096s
Jun 14 16:30:16.420: INFO: Pod "pvc-snapshottable-tester-24fk9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 21.386772482s
STEP: Saw pod success
Jun 14 16:30:16.420: INFO: Pod "pvc-snapshottable-tester-24fk9" satisfied condition "Succeeded or Failed"
Jun 14 16:30:16.672: INFO: Pod pvc-snapshottable-tester-24fk9 has the following logs: 
Jun 14 16:30:16.672: INFO: Deleting pod "pvc-snapshottable-tester-24fk9" in namespace "snapshotting-622"
Jun 14 16:30:16.806: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-24fk9" to be fully deleted
Jun 14 16:30:16.931: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [ebs.csi.aws.com8c9gw] to have phase Bound
Jun 14 16:30:17.056: INFO: PersistentVolumeClaim ebs.csi.aws.com8c9gw found and phase=Bound (125.005644ms)
STEP: [init] checking the claim
... skipping 34 lines ...
[It] should check snapshot fields, check restore correctly works after modifying source data, check deletion
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:243
STEP: checking the snapshot
STEP: checking the SnapshotContent
STEP: Modifying source data test
STEP: modifying the data in the source PVC
Jun 14 16:30:32.493: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-bhm4r" in namespace "snapshotting-622" to be "Succeeded or Failed"
Jun 14 16:30:32.618: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 124.72616ms
Jun 14 16:30:34.744: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 2.251090089s
Jun 14 16:30:36.870: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 4.377323078s
Jun 14 16:30:38.996: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 6.502829472s
Jun 14 16:30:41.121: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 8.628315659s
Jun 14 16:30:43.249: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 10.756523439s
Jun 14 16:30:45.375: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 12.882626303s
Jun 14 16:30:47.502: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 15.009561397s
Jun 14 16:30:49.628: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Pending", Reason="", readiness=false. Elapsed: 17.135111756s
Jun 14 16:30:51.754: INFO: Pod "pvc-snapshottable-data-tester-bhm4r": Phase="Succeeded", Reason="", readiness=false. Elapsed: 19.261156522s
STEP: Saw pod success
Jun 14 16:30:51.754: INFO: Pod "pvc-snapshottable-data-tester-bhm4r" satisfied condition "Succeeded or Failed"
Jun 14 16:30:52.006: INFO: Pod pvc-snapshottable-data-tester-bhm4r has the following logs: 
Jun 14 16:30:52.006: INFO: Deleting pod "pvc-snapshottable-data-tester-bhm4r" in namespace "snapshotting-622"
Jun 14 16:30:52.136: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-bhm4r" to be fully deleted
STEP: creating a pvc from the snapshot
STEP: starting a pod to use the claim
Jun 14 16:31:14.774: INFO: Running '/usr/local/bin/kubectl --server=https://api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=snapshotting-622 exec restored-pvc-tester-v67s9 --namespace=snapshotting-622 -- cat /mnt/test/data'
... skipping 224 lines ...
    /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:108
      
      /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:225
        should check snapshot fields, check restore correctly works after modifying source data, check deletion [It]
        /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:243

        Unexpected error:
            <exec.CodeExitError>: {
                Err: {
                    s: "error running /usr/local/bin/kubectl --server=https://api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=snapshotting-622 exec restored-pvc-tester-v67s9 --namespace=snapshotting-622 -- cat /mnt/test/data:\nCommand stdout:\n\nstderr:\ncat: can't open '/mnt/test/data': No such file or directory\ncommand terminated with exit code 1\n\nerror:\nexit status 1",
                },
                Code: 1,
            }
            error running /usr/local/bin/kubectl --server=https://api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=snapshotting-622 exec restored-pvc-tester-v67s9 --namespace=snapshotting-622 -- cat /mnt/test/data:
            Command stdout:
            
            stderr:
            cat: can't open '/mnt/test/data': No such file or directory
            command terminated with exit code 1
            
            error:
            exit status 1
        occurred

        /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/util.go:602
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath
... skipping 140 lines ...
Jun 14 16:32:20.831: INFO: Waiting for pod aws-client to disappear
Jun 14 16:32:20.958: INFO: Pod aws-client no longer exists
STEP: cleaning the environment after aws
STEP: Deleting pv and pvc
Jun 14 16:32:20.958: INFO: Deleting PersistentVolumeClaim "pvc-vnqpf"
Jun 14 16:32:21.086: INFO: Deleting PersistentVolume "aws-6z27v"
Jun 14 16:32:21.955: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-09068ecd0cbf4fc96", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-09068ecd0cbf4fc96 is currently attached to i-0fd2e5eaedfcde5eb
	status code: 400, request id: bf98b658-5264-4c0e-a40a-691af551b6e3
Jun 14 16:32:27.840: INFO: Couldn't delete PD "aws://ap-northeast-1a/vol-09068ecd0cbf4fc96", sleeping 5s: error deleting EBS volumes: VolumeInUse: Volume vol-09068ecd0cbf4fc96 is currently attached to i-0fd2e5eaedfcde5eb
	status code: 400, request id: 02ca074c-0b8c-40a1-b118-727fc0bf7316
Jun 14 16:32:33.654: INFO: Successfully deleted PD "aws://ap-northeast-1a/vol-09068ecd0cbf4fc96".
[AfterEach] [Testpattern: Pre-provisioned PV (ext4)] volumes
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:186
Jun 14 16:32:33.654: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "volume-8812" for this suite.
... skipping 22 lines ...
Jun 14 16:29:17.689: INFO: Creating resource for dynamic PV
Jun 14 16:29:17.689: INFO: Using claimSize:1Gi, test suite supported size:{ 1Mi}, driver(ebs.csi.aws.com) supported size:{ 1Mi} 
STEP: creating a StorageClass snapshotting-5343-e2e-sch668j
STEP: creating a claim
Jun 14 16:29:17.823: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
STEP: [init] starting a pod to use the claim
Jun 14 16:29:18.249: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-tbwcv" in namespace "snapshotting-5343" to be "Succeeded or Failed"
Jun 14 16:29:18.381: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 132.214246ms
Jun 14 16:29:20.518: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.269014045s
Jun 14 16:29:22.652: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.402854908s
Jun 14 16:29:24.785: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.536457867s
Jun 14 16:29:26.919: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.669929788s
Jun 14 16:29:29.053: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.80407818s
Jun 14 16:29:31.188: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 12.939299646s
Jun 14 16:29:33.322: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 15.073338515s
Jun 14 16:29:35.456: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 17.207562205s
Jun 14 16:29:37.589: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 19.340085178s
Jun 14 16:29:39.722: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Pending", Reason="", readiness=false. Elapsed: 21.473401164s
Jun 14 16:29:41.860: INFO: Pod "pvc-snapshottable-tester-tbwcv": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.611077277s
STEP: Saw pod success
Jun 14 16:29:41.860: INFO: Pod "pvc-snapshottable-tester-tbwcv" satisfied condition "Succeeded or Failed"
Jun 14 16:29:42.128: INFO: Pod pvc-snapshottable-tester-tbwcv has the following logs: 
Jun 14 16:29:42.128: INFO: Deleting pod "pvc-snapshottable-tester-tbwcv" in namespace "snapshotting-5343"
Jun 14 16:29:42.278: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-tbwcv" to be fully deleted
Jun 14 16:29:42.413: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [ebs.csi.aws.comdwwxz] to have phase Bound
Jun 14 16:29:42.546: INFO: PersistentVolumeClaim ebs.csi.aws.comdwwxz found and phase=Bound (133.531816ms)
STEP: [init] checking the claim
... skipping 25 lines ...
[It] should check snapshot fields, check restore correctly works after modifying source data, check deletion
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:243
STEP: checking the snapshot
STEP: checking the SnapshotContent
STEP: Modifying source data test
STEP: modifying the data in the source PVC
Jun 14 16:30:15.994: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-94rvk" in namespace "snapshotting-5343" to be "Succeeded or Failed"
Jun 14 16:30:16.123: INFO: Pod "pvc-snapshottable-data-tester-94rvk": Phase="Pending", Reason="", readiness=false. Elapsed: 129.131447ms
Jun 14 16:30:18.252: INFO: Pod "pvc-snapshottable-data-tester-94rvk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.258403316s
Jun 14 16:30:20.383: INFO: Pod "pvc-snapshottable-data-tester-94rvk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.389609901s
Jun 14 16:30:22.514: INFO: Pod "pvc-snapshottable-data-tester-94rvk": Phase="Succeeded", Reason="", readiness=false. Elapsed: 6.520181811s
STEP: Saw pod success
Jun 14 16:30:22.514: INFO: Pod "pvc-snapshottable-data-tester-94rvk" satisfied condition "Succeeded or Failed"
Jun 14 16:30:22.774: INFO: Pod pvc-snapshottable-data-tester-94rvk has the following logs: 
Jun 14 16:30:22.774: INFO: Deleting pod "pvc-snapshottable-data-tester-94rvk" in namespace "snapshotting-5343"
Jun 14 16:30:22.909: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-94rvk" to be fully deleted
STEP: creating a pvc from the snapshot
STEP: starting a pod to use the claim
Jun 14 16:31:33.565: INFO: Running '/usr/local/bin/kubectl --server=https://api.e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --kubeconfig=/root/.kube/config --namespace=snapshotting-5343 exec restored-pvc-tester-wglr4 --namespace=snapshotting-5343 -- cat /mnt/test/data'
... skipping 30 lines ...
Jun 14 16:31:59.241: INFO: volumesnapshotcontents snapcontent-5711f580-f6e5-486e-a1e1-9c1f3b73b5e4 has been found and is not deleted
Jun 14 16:32:00.377: INFO: volumesnapshotcontents snapcontent-5711f580-f6e5-486e-a1e1-9c1f3b73b5e4 has been found and is not deleted
Jun 14 16:32:01.514: INFO: volumesnapshotcontents snapcontent-5711f580-f6e5-486e-a1e1-9c1f3b73b5e4 has been found and is not deleted
Jun 14 16:32:02.645: INFO: volumesnapshotcontents snapcontent-5711f580-f6e5-486e-a1e1-9c1f3b73b5e4 has been found and is not deleted
Jun 14 16:32:03.779: INFO: volumesnapshotcontents snapcontent-5711f580-f6e5-486e-a1e1-9c1f3b73b5e4 has been found and is not deleted
Jun 14 16:32:04.909: INFO: volumesnapshotcontents snapcontent-5711f580-f6e5-486e-a1e1-9c1f3b73b5e4 has been found and is not deleted
Jun 14 16:32:05.910: INFO: WaitUntil failed after reaching the timeout 30s
[AfterEach] volume snapshot controller
  /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:221
Jun 14 16:32:06.049: INFO: Pod restored-pvc-tester-wglr4 has the following logs: 
Jun 14 16:32:06.049: INFO: Deleting pod "restored-pvc-tester-wglr4" in namespace "snapshotting-5343"
Jun 14 16:32:06.180: INFO: Wait up to 5m0s for pod "restored-pvc-tester-wglr4" to be fully deleted
Jun 14 16:32:40.439: INFO: deleting claim "snapshotting-5343"/"pvc-smfn9"
... skipping 32 lines ...
        /tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:243
------------------------------


Summarizing 1 Failure:

[Fail] External Storage [Driver: ebs.csi.aws.com] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  [It] should check snapshot fields, check restore correctly works after modifying source data, check deletion 
/tmp/kops.Ngif65n0h/vendor/k8s.io/kubernetes/test/e2e/framework/util.go:602

Ran 141 of 485 Specs in 599.105 seconds
FAIL! -- 140 Passed | 1 Failed | 0 Pending | 344 Skipped


Ginkgo ran 1 suite in 11m17.564241455s
Test Suite Failed
+ kops-finish
+ kubetest2 kops -v=2 --cloud-provider=aws --cluster-name=e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --admin-access= --kops-binary-path=/home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops --down
I0614 16:32:49.707353   27855 app.go:59] RunDir for this run: "/logs/artifacts/6682a14c-cd2a-11eb-ab6f-62c732df09e9"
I0614 16:32:49.707509   27855 app.go:90] ID for this run: "6682a14c-cd2a-11eb-ab6f-62c732df09e9"
I0614 16:32:49.707536   27855 dumplogs.go:38] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops toolbox dump --name e2e-f9b0e6b3f9-a6fg1.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I0614 16:32:49.730212   27861 featureflag.go:167] FeatureFlag "SpecOverrideFlag"=true
... skipping 1617 lines ...