This job view page is being replaced by Spyglass soon. Check out the new job view.
PRhakman: [WIP] Experiment with IPv6
ResultABORTED
Tests 0 failed / 0 succeeded
Started2021-06-28 10:30
Elapsed26m13s
Revisioncb64e66c8b69462ef14a38f8e7a992e88e9d0229
Refs 11874

No Test Failures!


Error lines from build-log.txt

... skipping 472 lines ...
I0628 10:35:15.194176    4363 dumplogs.go:38] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops toolbox dump --name e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io --dir /logs/artifacts --private-key /etc/aws-ssh/aws-ssh-private --ssh-user ubuntu
I0628 10:35:15.211031   11866 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0628 10:35:15.211133   11866 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0628 10:35:15.211140   11866 featureflag.go:165] FeatureFlag "AWSIPv6"=true

Cluster.kops.k8s.io "e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io" not found
W0628 10:35:15.700908    4363 down.go:34] Dumping cluster logs at the start of Down() failed: exit status 1
I0628 10:35:15.700953    4363 down.go:48] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops delete cluster --name e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io --yes
I0628 10:35:15.716880   11876 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0628 10:35:15.717001   11876 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0628 10:35:15.717006   11876 featureflag.go:165] FeatureFlag "AWSIPv6"=true

error reading cluster configuration: Cluster.kops.k8s.io "e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io" not found
I0628 10:35:16.222226    4363 http.go:37] curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip
2021/06/28 10:35:16 failed to get external ip from metadata service: http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip returned 404
I0628 10:35:16.229119    4363 http.go:37] curl https://ip.jsb.workers.dev
I0628 10:35:16.317180    4363 up.go:144] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops create cluster --name e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io --cloud aws --kubernetes-version https://storage.googleapis.com/kubernetes-release-dev/ci/v1.22.0-beta.0.151+89675101d0a2ef --ssh-public-key /etc/aws-ssh/aws-ssh-public --override cluster.spec.nodePortAccess=0.0.0.0/0 --yes --image=099720109477/ubuntu/images/hvm-ssd/ubuntu-hirsute-21.04-amd64-server-20210622.1 --channel=alpha --networking=calico --container-runtime=containerd --ipv6 --override=cluster.spec.api.loadBalancer.type=Public --override=cluster.spec.api.loadBalancer.class=Network --override=cluster.spec.api.loadBalancer.useForInternalApi=true --override=cluster.spec.cloudControllerManager.cloudProvider=aws --override=cluster.spec.cloudControllerManager.image=hakman/cloud-controller-manager:ipv6-1 --override=cluster.spec.nonMasqueradeCIDR=fd00:10:96::/64 --override=cluster.spec.kubeDNS.upstreamNameservers=2620:119:53::53 --admin-access 34.134.51.8/32 --master-count 1 --master-volume-size 48 --node-count 4 --node-volume-size 48 --zones us-east-1a --master-size c5.large
I0628 10:35:16.335124   11886 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0628 10:35:16.335227   11886 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0628 10:35:16.335232   11886 featureflag.go:165] FeatureFlag "AWSIPv6"=true
I0628 10:35:16.383015   11886 create_cluster.go:739] Using SSH public key: /etc/aws-ssh/aws-ssh-public
... skipping 33 lines ...
I0628 10:35:34.103912    4363 up.go:181] /home/prow/go/src/k8s.io/kops/bazel-bin/cmd/kops/linux-amd64/kops validate cluster --name e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io --count 10 --wait 15m0s
I0628 10:35:34.121120   11907 featureflag.go:165] FeatureFlag "SpecOverrideFlag"=true
I0628 10:35:34.121553   11907 featureflag.go:165] FeatureFlag "AlphaAllowGCE"=true
I0628 10:35:34.121631   11907 featureflag.go:165] FeatureFlag "AWSIPv6"=true
Validating cluster e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io

W0628 10:35:35.124320   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:35:45.154358   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:35:55.182265   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:36:05.212054   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:36:15.257587   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:36:25.302975   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:36:35.334337   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:36:45.369924   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:36:55.398911   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:37:05.445265   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:37:15.473107   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:37:25.500521   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:37:35.530183   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:37:45.562999   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:37:55.596445   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:38:05.616093   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:38:15.636240   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:38:25.671875   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:38:35.702868   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:38:45.752514   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:38:55.798292   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:39:05.826735   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:39:15.853018   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
W0628 10:39:55.887323   11907 validate_cluster.go:173] (will retry): unexpected error during validation: error listing nodes: Get "https://api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io/api/v1/nodes": dial tcp 35.170.183.98:443: i/o timeout
W0628 10:40:05.909378   11907 validate_cluster.go:173] (will retry): unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.e2e-786ab8c4d7-b7ef6.test-cncf-aws.k8s.io on 10.63.240.10:53: no such host
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

NODE STATUS
... skipping 6 lines ...
Machine	i-0123d690b9613c5cc				machine "i-0123d690b9613c5cc" has not yet joined cluster
Machine	i-04244886871cef572				machine "i-04244886871cef572" has not yet joined cluster
Machine	i-066e078566bae283f				machine "i-066e078566bae283f" has not yet joined cluster
Pod	kube-system/coredns-autoscaler-6f594f4c58-zdw5t	system-cluster-critical pod "coredns-autoscaler-6f594f4c58-zdw5t" is pending
Pod	kube-system/coredns-f45c4bf76-chghs		system-cluster-critical pod "coredns-f45c4bf76-chghs" is pending

Validation Failed
W0628 10:40:17.339151   11907 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 13 lines ...
Pod	kube-system/calico-node-hmph7			system-node-critical pod "calico-node-hmph7" is pending
Pod	kube-system/coredns-autoscaler-6f594f4c58-zdw5t	system-cluster-critical pod "coredns-autoscaler-6f594f4c58-zdw5t" is pending
Pod	kube-system/coredns-f45c4bf76-chghs		system-cluster-critical pod "coredns-f45c4bf76-chghs" is pending
Pod	kube-system/ebs-csi-node-b8hv7			system-node-critical pod "ebs-csi-node-b8hv7" is pending
Pod	kube-system/ebs-csi-node-n67w2			system-node-critical pod "ebs-csi-node-n67w2" is pending

Validation Failed
W0628 10:40:28.294086   11907 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 15 lines ...
Pod	kube-system/calico-node-lxchz			system-node-critical pod "calico-node-lxchz" is pending
Pod	kube-system/coredns-autoscaler-6f594f4c58-zdw5t	system-cluster-critical pod "coredns-autoscaler-6f594f4c58-zdw5t" is pending
Pod	kube-system/coredns-f45c4bf76-chghs		system-cluster-critical pod "coredns-f45c4bf76-chghs" is pending
Pod	kube-system/ebs-csi-node-pgq9s			system-node-critical pod "ebs-csi-node-pgq9s" is pending
Pod	kube-system/ebs-csi-node-s4qvb			system-node-critical pod "ebs-csi-node-s4qvb" is pending

Validation Failed
W0628 10:40:39.497742   11907 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 12 lines ...
Pod	kube-system/calico-node-jbbsg				system-node-critical pod "calico-node-jbbsg" is not ready (calico-node)
Pod	kube-system/calico-node-lxchz				system-node-critical pod "calico-node-lxchz" is not ready (calico-node)
Pod	kube-system/coredns-f45c4bf76-4hfp2			system-cluster-critical pod "coredns-f45c4bf76-4hfp2" is not ready (coredns)
Pod	kube-system/coredns-f45c4bf76-chghs			system-cluster-critical pod "coredns-f45c4bf76-chghs" is not ready (coredns)
Pod	kube-system/kube-proxy-ip-172-20-62-25.ec2.internal	system-node-critical pod "kube-proxy-ip-172-20-62-25.ec2.internal" is pending

Validation Failed
W0628 10:40:50.768749   11907 validate_cluster.go:221] (will retry): cluster not yet healthy
INSTANCE GROUPS
NAME			ROLE	MACHINETYPE	MIN	MAX	SUBNETS
master-us-east-1a	Master	c5.large	1	1	us-east-1a
nodes-us-east-1a	Node	t3.medium	4	4	us-east-1a

... skipping 56 lines ...