Recent runs || View in Spyglass
Result | FAILURE |
Tests | 1 failed / 6 succeeded |
Started | |
Elapsed | 32m8s |
Revision | main |
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=capi\-e2e\s\[It\]\sWhen\supgrading\sa\sworkload\scluster\susing\sClusterClass\sand\stesting\sK8S\sconformance\s\[Conformance\]\s\[K8s\-Upgrade\]\s\[ClusterClass\]\sShould\screate\sand\supgrade\sa\sworkload\scluster\sand\seventually\srun\skubetest$'
[FAILED] Timed out after 390.091s. Timed out waiting for all MachinePool k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-mp-0 instances to be upgraded to Kubernetes version v1.22.17 Error: function returned error: old version instances remain <*fmt.wrapError | 0xc001a12080>: { msg: "function returned error: old version instances remain", err: <*errors.fundamental | 0xc002708258>{ msg: "old version instances remain", stack: [0x181165a, 0x4dad65, 0x4da25c, 0x88691a, 0x887822, 0x88518d, 0x1811464, 0x180fb85, 0x1cb830a, 0x8630bb, 0x8771b8, 0x4704e1], }, } In [It] at: /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 @ 01/10/23 12:01:53.435from junit.e2e_suite.1.xml
clusterclass.cluster.x-k8s.io/quick-start created dockerclustertemplate.infrastructure.cluster.x-k8s.io/quick-start-cluster created kubeadmcontrolplanetemplate.controlplane.cluster.x-k8s.io/quick-start-control-plane created dockermachinetemplate.infrastructure.cluster.x-k8s.io/quick-start-control-plane created dockermachinetemplate.infrastructure.cluster.x-k8s.io/quick-start-default-worker-machinetemplate created kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/quick-start-default-worker-bootstraptemplate created configmap/cni-k8s-upgrade-and-conformance-ie3lk6-crs-0 created clusterresourceset.addons.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-crs-0 created kubeadmconfig.bootstrap.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-mp-0-config created kubeadmconfig.bootstrap.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-mp-0-config-cgroupfs created cluster.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6 created machinepool.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-mp-0 created dockermachinepool.infrastructure.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-dmp-0 created Failed to get logs for Machine k8s-upgrade-and-conformance-ie3lk6-md-0-685br-774f795f5d-mj9z7, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 Failed to get logs for Machine k8s-upgrade-and-conformance-ie3lk6-md-0-685br-774f795f5d-nrh82, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 Failed to get logs for Machine k8s-upgrade-and-conformance-ie3lk6-ttpcr-cqhzz, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 Failed to get logs for MachinePool k8s-upgrade-and-conformance-ie3lk6-mp-0, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 > Enter [BeforeEach] When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:84 @ 01/10/23 11:48:49.455 STEP: Creating a namespace for hosting the "k8s-upgrade-and-conformance" test spec - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/common.go:51 @ 01/10/23 11:48:49.455 INFO: Creating namespace k8s-upgrade-and-conformance-x32go1 INFO: Creating event watcher for namespace "k8s-upgrade-and-conformance-x32go1" < Exit [BeforeEach] When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:84 @ 01/10/23 11:48:49.489 (34ms) > Enter [It] Should create and upgrade a workload cluster and eventually run kubetest - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 @ 01/10/23 11:48:49.489 STEP: Creating a workload cluster - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:119 @ 01/10/23 11:48:49.489 INFO: Creating the workload cluster with name "k8s-upgrade-and-conformance-ie3lk6" using the "upgrades-cgroupfs" template (Kubernetes v1.21.14, 1 control-plane machines, 2 worker machines) INFO: Getting the cluster template yaml INFO: clusterctl config cluster k8s-upgrade-and-conformance-ie3lk6 --infrastructure (default) --kubernetes-version v1.21.14 --control-plane-machine-count 1 --worker-machine-count 2 --flavor upgrades-cgroupfs INFO: Applying the cluster template yaml to the cluster INFO: Waiting for the cluster infrastructure to be provisioned STEP: Waiting for cluster to enter the provisioned phase - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/cluster_helpers.go:134 @ 01/10/23 11:48:52.308 INFO: Waiting for control plane to be initialized INFO: Waiting for the first control plane machine managed by k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-ttpcr to be provisioned STEP: Waiting for one control plane node to exist - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/controlplane_helpers.go:133 @ 01/10/23 11:49:02.38 INFO: Waiting for control plane to be ready INFO: Waiting for control plane k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-ttpcr to be ready (implies underlying nodes to be ready as well) STEP: Waiting for the control plane to be ready - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/controlplane_helpers.go:165 @ 01/10/23 11:49:42.457 STEP: Checking all the control plane machines are in the expected failure domains - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/controlplane_helpers.go:196 @ 01/10/23 11:49:52.473 INFO: Waiting for the machine deployments to be provisioned STEP: Waiting for the workload nodes to exist - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinedeployment_helpers.go:102 @ 01/10/23 11:49:52.513 STEP: Checking all the machines controlled by k8s-upgrade-and-conformance-ie3lk6-md-0-685br are in the "fd4" failure domain - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/ginkgoextensions/output.go:35 @ 01/10/23 11:50:32.666 INFO: Waiting for the machine pools to be provisioned STEP: Waiting for the machine pool workload nodes - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:79 @ 01/10/23 11:50:32.745 STEP: Upgrading the Cluster topology - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:144 @ 01/10/23 11:51:52.827 INFO: Patching the new Kubernetes version to Cluster topology INFO: Waiting for control-plane machines to have the upgraded Kubernetes version STEP: Ensuring all control-plane machines have upgraded kubernetes version v1.22.17 - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/ginkgoextensions/output.go:35 @ 01/10/23 11:51:52.93 INFO: Waiting for kube-proxy to have the upgraded Kubernetes version STEP: Ensuring kube-proxy has the correct image - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/daemonset_helpers.go:41 @ 01/10/23 11:53:33.11 INFO: Waiting for CoreDNS to have the upgraded image tag STEP: Ensuring CoreDNS has the correct image - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/deployment_helpers.go:276 @ 01/10/23 11:53:43.12 INFO: Waiting for etcd to have the upgraded image tag INFO: Waiting for Kubernetes versions of machines in MachineDeployment k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-md-0-685br to be upgraded to v1.22.17 INFO: Ensuring all MachineDeployment Machines have upgraded kubernetes version v1.22.17 STEP: Upgrading the machinepool instances - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:203 @ 01/10/23 11:55:23.246 INFO: Patching the new Kubernetes version to Machine Pool k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-mp-0 INFO: Waiting for Kubernetes versions of machines in MachinePool k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-mp-0 to be upgraded from v1.21.14 to v1.22.17 INFO: Ensuring all MachinePool Instances have upgraded kubernetes version v1.22.17 Automatically polling progress: When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] Should create and upgrade a workload cluster and eventually run kubetest (Spec Runtime: 10m0.035s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 In [It] (Node Runtime: 10m0.001s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 At [By Step] Upgrading the machinepool instances (Step Runtime: 3m26.243s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:203 Spec Goroutine goroutine 22332 [select] k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x266fe20, 0xc000138000}, 0xc000d66dc8, 0x11992aa?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:660 k8s.io/apimachinery/pkg/util/wait.poll({0x266fe20, 0xc000138000}, 0x48?, 0x1198065?, 0x50?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:596 k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x266fe20, 0xc000138000}, 0x2661588?, 0xc0011fe898?, 0x40fac7?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:528 k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc000f111c0?, 0x2661588?, 0x3900d18?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:514 sigs.k8s.io/cluster-api/test/framework.getMachinePoolInstanceVersions({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0003b6700}, {0xc002299dd0, 0x22}, 0xc000801500}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:289 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded.func1() /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:250 reflect.Value.call({0x1e8a900?, 0xc001a64ba0?, 0x60?}, {0x22c9139, 0x4}, {0x3900d18, 0x0, 0x0?}) /usr/local/go/src/reflect/value.go:584 reflect.Value.Call({0x1e8a900?, 0xc001a64ba0?, 0x1?}, {0x3900d18?, 0x0?, 0xc001515000?}) /usr/local/go/src/reflect/value.go:368 github.com/onsi/gomega/internal.(*AsyncAssertion).buildActualPoller.func3() /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:281 github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0003b6770, {0x26616a8?, 0xc001f93540}, 0x1, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:440 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0003b6770, {0x26616a8, 0xc001f93540}, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:121 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0000f0690}, {0x7f77c4390f48, 0xc0003b6700}, 0xc001222340, {0xc0000542be, 0x8}, 0x2, ...}, ...) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 sigs.k8s.io/cluster-api/test/framework.UpgradeMachinePoolAndWait({0x266fde8?, 0xc0009ac1c0}, {{0x267e4e8, 0xc001d51a40}, 0xc001222340, {0xc0000542be, 0x8}, {0xc000129620, 0x1, 0x1}, ...}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:176 > sigs.k8s.io/cluster-api/test/e2e.ClusterUpgradeConformanceSpec.func2() /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:204 | if len(clusterResources.MachinePools) > 0 && workerMachineCount > 0 { | By("Upgrading the machinepool instances") > framework.UpgradeMachinePoolAndWait(ctx, framework.UpgradeMachinePoolAndWaitInput{ | ClusterProxy: input.BootstrapClusterProxy, | Cluster: clusterResources.Cluster, github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0xc000ec95e0, 0xc0023bf830}) /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/node.go:459 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:854 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:841 Automatically polling progress: When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] Should create and upgrade a workload cluster and eventually run kubetest (Spec Runtime: 11m0.039s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 In [It] (Node Runtime: 11m0.005s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 At [By Step] Upgrading the machinepool instances (Step Runtime: 4m26.248s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:203 Spec Goroutine goroutine 22332 [select] k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x266fe20, 0xc000138000}, 0xc000648648, 0x11992aa?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:660 k8s.io/apimachinery/pkg/util/wait.poll({0x266fe20, 0xc000138000}, 0x48?, 0x1198065?, 0x50?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:596 k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x266fe20, 0xc000138000}, 0x2661588?, 0xc0011fe898?, 0x40fac7?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:528 k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc000f111c0?, 0x2661588?, 0x3900d18?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:514 sigs.k8s.io/cluster-api/test/framework.getMachinePoolInstanceVersions({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0003b6700}, {0xc002299dd0, 0x22}, 0xc000801500}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:289 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded.func1() /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:250 reflect.Value.call({0x1e8a900?, 0xc001a64ba0?, 0x60?}, {0x22c9139, 0x4}, {0x3900d18, 0x0, 0x0?}) /usr/local/go/src/reflect/value.go:584 reflect.Value.Call({0x1e8a900?, 0xc001a64ba0?, 0x1?}, {0x3900d18?, 0x0?, 0xc001515000?}) /usr/local/go/src/reflect/value.go:368 github.com/onsi/gomega/internal.(*AsyncAssertion).buildActualPoller.func3() /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:281 github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0003b6770, {0x26616a8?, 0xc001f93540}, 0x1, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:440 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0003b6770, {0x26616a8, 0xc001f93540}, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:121 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0000f0690}, {0x7f77c4390f48, 0xc0003b6700}, 0xc001222340, {0xc0000542be, 0x8}, 0x2, ...}, ...) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 sigs.k8s.io/cluster-api/test/framework.UpgradeMachinePoolAndWait({0x266fde8?, 0xc0009ac1c0}, {{0x267e4e8, 0xc001d51a40}, 0xc001222340, {0xc0000542be, 0x8}, {0xc000129620, 0x1, 0x1}, ...}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:176 > sigs.k8s.io/cluster-api/test/e2e.ClusterUpgradeConformanceSpec.func2() /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:204 | if len(clusterResources.MachinePools) > 0 && workerMachineCount > 0 { | By("Upgrading the machinepool instances") > framework.UpgradeMachinePoolAndWait(ctx, framework.UpgradeMachinePoolAndWaitInput{ | ClusterProxy: input.BootstrapClusterProxy, | Cluster: clusterResources.Cluster, github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0xc000ec95e0, 0xc0023bf830}) /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/node.go:459 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:854 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:841 Automatically polling progress: When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] Should create and upgrade a workload cluster and eventually run kubetest (Spec Runtime: 12m0.042s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 In [It] (Node Runtime: 12m0.007s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 At [By Step] Upgrading the machinepool instances (Step Runtime: 5m26.25s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:203 Spec Goroutine goroutine 22332 [select] k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x266fe20, 0xc000138000}, 0xc000648648, 0x11992aa?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:660 k8s.io/apimachinery/pkg/util/wait.poll({0x266fe20, 0xc000138000}, 0x48?, 0x1198065?, 0x50?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:596 k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x266fe20, 0xc000138000}, 0x2661588?, 0xc0011fe898?, 0x40fac7?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:528 k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc000f111c0?, 0x2661588?, 0x3900d18?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:514 sigs.k8s.io/cluster-api/test/framework.getMachinePoolInstanceVersions({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0003b6700}, {0xc002299dd0, 0x22}, 0xc000801500}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:289 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded.func1() /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:250 reflect.Value.call({0x1e8a900?, 0xc001a64ba0?, 0x60?}, {0x22c9139, 0x4}, {0x3900d18, 0x0, 0x0?}) /usr/local/go/src/reflect/value.go:584 reflect.Value.Call({0x1e8a900?, 0xc001a64ba0?, 0x1?}, {0x3900d18?, 0x0?, 0xc001515000?}) /usr/local/go/src/reflect/value.go:368 github.com/onsi/gomega/internal.(*AsyncAssertion).buildActualPoller.func3() /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:281 github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0003b6770, {0x26616a8?, 0xc001f93540}, 0x1, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:440 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0003b6770, {0x26616a8, 0xc001f93540}, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:121 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0000f0690}, {0x7f77c4390f48, 0xc0003b6700}, 0xc001222340, {0xc0000542be, 0x8}, 0x2, ...}, ...) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 sigs.k8s.io/cluster-api/test/framework.UpgradeMachinePoolAndWait({0x266fde8?, 0xc0009ac1c0}, {{0x267e4e8, 0xc001d51a40}, 0xc001222340, {0xc0000542be, 0x8}, {0xc000129620, 0x1, 0x1}, ...}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:176 > sigs.k8s.io/cluster-api/test/e2e.ClusterUpgradeConformanceSpec.func2() /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:204 | if len(clusterResources.MachinePools) > 0 && workerMachineCount > 0 { | By("Upgrading the machinepool instances") > framework.UpgradeMachinePoolAndWait(ctx, framework.UpgradeMachinePoolAndWaitInput{ | ClusterProxy: input.BootstrapClusterProxy, | Cluster: clusterResources.Cluster, github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0xc000ec95e0, 0xc0023bf830}) /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/node.go:459 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:854 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:841 Automatically polling progress: When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] Should create and upgrade a workload cluster and eventually run kubetest (Spec Runtime: 13m0.045s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 In [It] (Node Runtime: 13m0.011s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 At [By Step] Upgrading the machinepool instances (Step Runtime: 6m26.254s) /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:203 Spec Goroutine goroutine 22332 [select] k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x266fe20, 0xc000138000}, 0xc000648648, 0x11992aa?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:660 k8s.io/apimachinery/pkg/util/wait.poll({0x266fe20, 0xc000138000}, 0x48?, 0x1198065?, 0x50?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:596 k8s.io/apimachinery/pkg/util/wait.PollImmediateWithContext({0x266fe20, 0xc000138000}, 0x2661588?, 0xc0011fe898?, 0x40fac7?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:528 k8s.io/apimachinery/pkg/util/wait.PollImmediate(0xc000f111c0?, 0x2661588?, 0x3900d18?) /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.25.0/pkg/util/wait/wait.go:514 sigs.k8s.io/cluster-api/test/framework.getMachinePoolInstanceVersions({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0003b6700}, {0xc002299dd0, 0x22}, 0xc000801500}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:289 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded.func1() /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:250 reflect.Value.call({0x1e8a900?, 0xc001a64ba0?, 0x60?}, {0x22c9139, 0x4}, {0x3900d18, 0x0, 0x0?}) /usr/local/go/src/reflect/value.go:584 reflect.Value.Call({0x1e8a900?, 0xc001a64ba0?, 0x1?}, {0x3900d18?, 0x0?, 0xc001515000?}) /usr/local/go/src/reflect/value.go:368 github.com/onsi/gomega/internal.(*AsyncAssertion).buildActualPoller.func3() /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:281 github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc0003b6770, {0x26616a8?, 0xc001f93540}, 0x1, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:440 github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc0003b6770, {0x26616a8, 0xc001f93540}, {0xc0012c0cf0, 0x3, 0x3}) /home/prow/go/pkg/mod/github.com/onsi/gomega@v1.24.2/internal/async_assertion.go:121 sigs.k8s.io/cluster-api/test/framework.WaitForMachinePoolInstancesToBeUpgraded({0x266fde8?, 0xc0009ac1c0}, {{0x7f77c4390f48, 0xc0000f0690}, {0x7f77c4390f48, 0xc0003b6700}, 0xc001222340, {0xc0000542be, 0x8}, 0x2, ...}, ...) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 sigs.k8s.io/cluster-api/test/framework.UpgradeMachinePoolAndWait({0x266fde8?, 0xc0009ac1c0}, {{0x267e4e8, 0xc001d51a40}, 0xc001222340, {0xc0000542be, 0x8}, {0xc000129620, 0x1, 0x1}, ...}) /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:176 > sigs.k8s.io/cluster-api/test/e2e.ClusterUpgradeConformanceSpec.func2() /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:204 | if len(clusterResources.MachinePools) > 0 && workerMachineCount > 0 { | By("Upgrading the machinepool instances") > framework.UpgradeMachinePoolAndWait(ctx, framework.UpgradeMachinePoolAndWaitInput{ | ClusterProxy: input.BootstrapClusterProxy, | Cluster: clusterResources.Cluster, github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0xc000ec95e0, 0xc0023bf830}) /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/node.go:459 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:854 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:841 [FAILED] Timed out after 390.091s. Timed out waiting for all MachinePool k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-mp-0 instances to be upgraded to Kubernetes version v1.22.17 Error: function returned error: old version instances remain <*fmt.wrapError | 0xc001a12080>: { msg: "function returned error: old version instances remain", err: <*errors.fundamental | 0xc002708258>{ msg: "old version instances remain", stack: [0x181165a, 0x4dad65, 0x4da25c, 0x88691a, 0x887822, 0x88518d, 0x1811464, 0x180fb85, 0x1cb830a, 0x8630bb, 0x8771b8, 0x4704e1], }, } In [It] at: /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 @ 01/10/23 12:01:53.435 < Exit [It] Should create and upgrade a workload cluster and eventually run kubetest - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 @ 01/10/23 12:01:53.435 (13m3.946s) > Enter [AfterEach] When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:242 @ 01/10/23 12:01:53.435 STEP: Dumping logs from the "k8s-upgrade-and-conformance-ie3lk6" workload cluster - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/common.go:51 @ 01/10/23 12:01:53.435 STEP: Dumping all the Cluster API resources in the "k8s-upgrade-and-conformance-x32go1" namespace - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/common.go:51 @ 01/10/23 12:01:56.527 STEP: Deleting cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6 - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/common.go:51 @ 01/10/23 12:01:56.863 STEP: Deleting cluster k8s-upgrade-and-conformance-ie3lk6 - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/ginkgoextensions/output.go:35 @ 01/10/23 12:01:56.887 INFO: Waiting for the Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6 to be deleted STEP: Waiting for cluster k8s-upgrade-and-conformance-ie3lk6 to be deleted - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/ginkgoextensions/output.go:35 @ 01/10/23 12:01:56.908 STEP: Deleting namespace used for hosting the "k8s-upgrade-and-conformance" test spec - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/common.go:51 @ 01/10/23 12:02:06.917 INFO: Deleting namespace k8s-upgrade-and-conformance-x32go1 < Exit [AfterEach] When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] - /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:242 @ 01/10/23 12:02:06.937 (13.503s)
Filter through log files | View test history on testgrid
capi-e2e [SynchronizedAfterSuite]
capi-e2e [SynchronizedAfterSuite]
capi-e2e [SynchronizedAfterSuite]
capi-e2e [SynchronizedBeforeSuite]
capi-e2e [SynchronizedBeforeSuite]
capi-e2e [SynchronizedBeforeSuite]
capi-e2e [It] When following the Cluster API quick-start [PR-Blocking] Should create a workload cluster
capi-e2e [It] When following the Cluster API quick-start with ClusterClass [PR-Informing] [ClusterClass] Should create a workload cluster
capi-e2e [It] When following the Cluster API quick-start with IPv6 [IPv6] [PR-Informing] Should create a workload cluster
capi-e2e [It] When following the Cluster API quick-start with Ignition Should create a workload cluster
capi-e2e [It] When testing Cluster API working on self-hosted clusters Should pivot the bootstrap cluster to a self-hosted cluster
capi-e2e [It] When testing Cluster API working on self-hosted clusters using ClusterClass [ClusterClass] Should pivot the bootstrap cluster to a self-hosted cluster
capi-e2e [It] When testing Cluster API working on self-hosted clusters using ClusterClass with a HA control plane [ClusterClass] Should pivot the bootstrap cluster to a self-hosted cluster
capi-e2e [It] When testing Cluster API working on single-node self-hosted clusters using ClusterClass [ClusterClass] Should pivot the bootstrap cluster to a self-hosted cluster
capi-e2e [It] When testing ClusterClass changes [ClusterClass] Should successfully rollout the managed topology upon changes to the ClusterClass
capi-e2e [It] When testing K8S conformance [Conformance] Should create a workload cluster and run kubetest
capi-e2e [It] When testing KCP adoption Should adopt up-to-date control plane Machines without modification
capi-e2e [It] When testing MachineDeployment rolling upgrades Should successfully upgrade Machines upon changes in relevant MachineDeployment fields
capi-e2e [It] When testing MachineDeployment scale out/in Should successfully scale a MachineDeployment up and down upon changes to the MachineDeployment replica count
capi-e2e [It] When testing MachinePools Should successfully create a cluster with machine pool machines
capi-e2e [It] When testing clusterctl upgrades (v0.3=>current) Should create a management cluster and then upgrade all the providers
capi-e2e [It] When testing clusterctl upgrades (v0.4=>current) Should create a management cluster and then upgrade all the providers
capi-e2e [It] When testing clusterctl upgrades (v1.2=>current) Should create a management cluster and then upgrade all the providers
capi-e2e [It] When testing clusterctl upgrades (v1.3=>current) Should create a management cluster and then upgrade all the providers
capi-e2e [It] When testing clusterctl upgrades using ClusterClass (v1.2=>current) [ClusterClass] Should create a management cluster and then upgrade all the providers
capi-e2e [It] When testing clusterctl upgrades using ClusterClass (v1.3=>current) [ClusterClass] Should create a management cluster and then upgrade all the providers
capi-e2e [It] When testing node drain timeout A node should be forcefully removed if it cannot be drained in time
capi-e2e [It] When testing unhealthy machines remediation Should successfully trigger KCP remediation
capi-e2e [It] When testing unhealthy machines remediation Should successfully trigger machine deployment remediation
capi-e2e [It] When upgrading a workload cluster using ClusterClass [ClusterClass] Should create and upgrade a workload cluster and eventually run kubetest
capi-e2e [It] When upgrading a workload cluster using ClusterClass with RuntimeSDK [PR-Informing] [ClusterClass] Should create, upgrade and delete a workload cluster
capi-e2e [It] When upgrading a workload cluster using ClusterClass with a HA control plane [ClusterClass] Should create and upgrade a workload cluster and eventually run kubetest
capi-e2e [It] When upgrading a workload cluster using ClusterClass with a HA control plane using scale-in rollout [ClusterClass] Should create and upgrade a workload cluster and eventually run kubetest
... skipping 615 lines ... Updating kustomize pull policy file for manager resources sed -i'' -e 's@imagePullPolicy: .*@imagePullPolicy: '"IfNotPresent"'@' ./test/extension/config/default/manager_pull_policy.yaml make[2]: Leaving directory '/home/prow/go/src/sigs.k8s.io/cluster-api' make[1]: Leaving directory '/home/prow/go/src/sigs.k8s.io/cluster-api' + KUBERNETES_VERSION_UPGRADE_TO="stable-1.22" resolved to "v1.22.17" + Pulling kindest/node:v1.22.17 Error response from daemon: manifest for kindest/node:v1.22.17 not found: manifest unknown: manifest unknown + image for Kuberentes v1.22.17 is not available in docker hub, trying local build KUBE_ROOT /home/prow/go/src/k8s.io/kubernetes + Checkout branch for Kubernetes v1.22.17 + checkout tag v1.22.17 Switched to a new branch 'v1.22.17-branch' + Setting version for Kubernetes build to v1.22.17 ... skipping 192 lines ... /home/prow/go/src/sigs.k8s.io/cluster-api/hack/tools/bin/kustomize-v4.5.2 build test/e2e/data/infrastructure-docker/v1beta1/main/cluster-template-ignition --load-restrictor LoadRestrictionsNone > test/e2e/data/infrastructure-docker/v1beta1/main/cluster-template-ignition.yaml mkdir -p test/e2e/data/test-extension /home/prow/go/src/sigs.k8s.io/cluster-api/hack/tools/bin/kustomize-v4.5.2 build test/extension/config/default > test/e2e/data/test-extension/deployment.yaml /home/prow/go/src/sigs.k8s.io/cluster-api/hack/tools/bin/ginkgo-v2.7.0 -v --trace -poll-progress-after=10m \ -poll-progress-interval=1m --tags=e2e --focus="\[K8s-Upgrade\]" \ --nodes=3 --timeout=2h --no-color=true \ --output-dir="/logs/artifacts" --junit-report="junit.e2e_suite.1.xml" --fail-fast /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e -- \ -e2e.artifacts-folder="/logs/artifacts" \ -e2e.config="/home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/config/docker.yaml" \ -e2e.skip-resource-cleanup=false -e2e.use-existing-cluster=false go: downloading github.com/onsi/gomega v1.24.2 go: downloading k8s.io/api v0.25.0 go: downloading k8s.io/apimachinery v0.25.0 ... skipping 453 lines ... /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/node.go:459 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:854 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:841 ------------------------------ • [FAILED] [797.483 seconds] When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] [It] Should create and upgrade a workload cluster and eventually run kubetest /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/cluster_upgrade.go:118 Captured StdOut/StdErr Output >> clusterclass.cluster.x-k8s.io/quick-start created dockerclustertemplate.infrastructure.cluster.x-k8s.io/quick-start-cluster created ... skipping 6 lines ... kubeadmconfig.bootstrap.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-mp-0-config created kubeadmconfig.bootstrap.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-mp-0-config-cgroupfs created cluster.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6 created machinepool.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-mp-0 created dockermachinepool.infrastructure.cluster.x-k8s.io/k8s-upgrade-and-conformance-ie3lk6-dmp-0 created Failed to get logs for Machine k8s-upgrade-and-conformance-ie3lk6-md-0-685br-774f795f5d-mj9z7, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 Failed to get logs for Machine k8s-upgrade-and-conformance-ie3lk6-md-0-685br-774f795f5d-nrh82, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 Failed to get logs for Machine k8s-upgrade-and-conformance-ie3lk6-ttpcr-cqhzz, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 Failed to get logs for MachinePool k8s-upgrade-and-conformance-ie3lk6-mp-0, Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6: exit status 2 << Captured StdOut/StdErr Output Timeline >> STEP: Creating a namespace for hosting the "k8s-upgrade-and-conformance" test spec @ 01/10/23 11:48:49.455 INFO: Creating namespace k8s-upgrade-and-conformance-x32go1 INFO: Creating event watcher for namespace "k8s-upgrade-and-conformance-x32go1" ... skipping 224 lines ... github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0xc000ec95e0, 0xc0023bf830}) /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/node.go:459 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3() /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:854 github.com/onsi/ginkgo/v2/internal.(*Suite).runNode /home/prow/go/pkg/mod/github.com/onsi/ginkgo/v2@v2.7.0/internal/suite.go:841 [FAILED] in [It] - /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 @ 01/10/23 12:01:53.435 STEP: Dumping logs from the "k8s-upgrade-and-conformance-ie3lk6" workload cluster @ 01/10/23 12:01:53.435 STEP: Dumping all the Cluster API resources in the "k8s-upgrade-and-conformance-x32go1" namespace @ 01/10/23 12:01:56.527 STEP: Deleting cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6 @ 01/10/23 12:01:56.863 STEP: Deleting cluster k8s-upgrade-and-conformance-ie3lk6 @ 01/10/23 12:01:56.887 INFO: Waiting for the Cluster k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6 to be deleted STEP: Waiting for cluster k8s-upgrade-and-conformance-ie3lk6 to be deleted @ 01/10/23 12:01:56.908 STEP: Deleting namespace used for hosting the "k8s-upgrade-and-conformance" test spec @ 01/10/23 12:02:06.917 INFO: Deleting namespace k8s-upgrade-and-conformance-x32go1 << Timeline [FAILED] Timed out after 390.091s. Timed out waiting for all MachinePool k8s-upgrade-and-conformance-x32go1/k8s-upgrade-and-conformance-ie3lk6-mp-0 instances to be upgraded to Kubernetes version v1.22.17 Error: function returned error: old version instances remain <*fmt.wrapError | 0xc001a12080>: { msg: "function returned error: old version instances remain", err: <*errors.fundamental | 0xc002708258>{ msg: "old version instances remain", stack: [0x181165a, 0x4dad65, 0x4da25c, 0x88691a, 0x887822, 0x88518d, 0x1811464, 0x180fb85, 0x1cb830a, 0x8630bb, 0x8771b8, 0x4704e1], }, } In [It] at: /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 @ 01/10/23 12:01:53.435 ... skipping 23 lines ... [ReportAfterSuite] PASSED [0.010 seconds] [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report autogenerated by Ginkgo ------------------------------ Summarizing 2 Failures: [FAIL] When upgrading a workload cluster using ClusterClass and testing K8S conformance [Conformance] [K8s-Upgrade] [ClusterClass] [It] Should create and upgrade a workload cluster and eventually run kubetest /home/prow/go/src/sigs.k8s.io/cluster-api/test/framework/machinepool_helpers.go:268 [INTERRUPTED] [SynchronizedAfterSuite] /home/prow/go/src/sigs.k8s.io/cluster-api/test/e2e/e2e_suite_test.go:169 Ran 1 of 28 Specs in 898.007 seconds FAIL! - Interrupted by Other Ginkgo Process -- 0 Passed | 1 Failed | 0 Pending | 27 Skipped Ginkgo ran 1 suite in 16m8.066091185s Test Suite Failed make: *** [Makefile:790: test-e2e] Error 1 WARNING: No swap limit support + EXIT_VALUE=2 + set +o xtrace Cleaning up after docker in docker. ================================================================================ Cleaning up after docker ... skipping 6 lines ...