This job view page is being replaced by Spyglass soon. Check out the new job view.
PRsbueringer: ✨ v0.3.x: KCP: block mgmt cluster updates to v1.22.0
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2021-07-28 16:01
Elapsed2m48s
Revision1744e2f37ac68cc5219f3900198b8a9593a8f1bd
Refs 5013

No Test Failures!


Error lines from build-log.txt

... skipping 785 lines ...
I0728 16:03:22.140213    8499 certwatcher.go:83] controller-runtime/certwatcher "msg"="Starting certificate watcher"  
I0728 16:03:22.702538    8499 kubeadmconfig_controller.go:223]  "msg"="Cluster infrastructure is not ready, waiting" "kind"="Machine" "kubeadmconfig"={"Namespace":"default","Name":"my-machine-config"} "name"="my-machine" "version"="109" 
•I0728 16:03:22.741332    8499 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  


Ran 1 of 1 Specs in 14.182 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestAPIs (14.18s)
PASS
ok  	sigs.k8s.io/cluster-api/bootstrap/kubeadm/controllers	60.298s
=== RUN   TestNewInitControlPlaneAdditionalFileEncodings
--- PASS: TestNewInitControlPlaneAdditionalFileEncodings (0.01s)
=== RUN   TestNewInitControlPlaneCommands
... skipping 310 lines ...
=== RUN   Test_clusterctlClient_GetProvidersConfig/Returns_default_providers_and_custom_providers_if_defined
--- PASS: Test_clusterctlClient_GetProvidersConfig (0.00s)
    --- PASS: Test_clusterctlClient_GetProvidersConfig/Returns_default_providers (0.00s)
    --- PASS: Test_clusterctlClient_GetProvidersConfig/Returns_default_providers_and_custom_providers_if_defined (0.00s)
=== RUN   Test_clusterctlClient_GetProviderComponents
=== RUN   Test_clusterctlClient_GetProviderComponents/Pass
=== RUN   Test_clusterctlClient_GetProviderComponents/Fail
--- PASS: Test_clusterctlClient_GetProviderComponents (0.00s)
    --- PASS: Test_clusterctlClient_GetProviderComponents/Pass (0.00s)
    --- PASS: Test_clusterctlClient_GetProviderComponents/Fail (0.00s)
=== RUN   Test_getComponentsByName_withEmptyVariables
--- PASS: Test_getComponentsByName_withEmptyVariables (0.00s)
=== RUN   Test_clusterctlClient_templateOptionsToVariables
=== RUN   Test_clusterctlClient_templateOptionsToVariables/pass_(using_KubernetesVersion_from_template_options)
=== RUN   Test_clusterctlClient_templateOptionsToVariables/pass_(using_KubernetesVersion_from_env_variables)
=== RUN   Test_clusterctlClient_templateOptionsToVariables/pass_(using_defaults_for_machine_counts)
... skipping 1398 lines ...
I0728 16:02:56.556198   10540 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="machinedeployment" 
I0728 16:02:56.556230   10540 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="machinedeployment" "worker count"=1
I0728 16:02:56.561702   10540 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"clusterName":"","bootstrap":{},"infrastructureRef":{}},"status":{"bootstrapReady":false,"infrastructureReady":false}}}
I0728 16:02:56.561782   10540 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"controlPlaneEndpoint":{"host":"","port":0}},"status":{"infrastructureReady":false,"controlPlaneInitialized":false}}}
I0728 16:02:56.561839   10540 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="machinehealthcheck" 
I0728 16:02:56.561886   10540 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="machinehealthcheck" "worker count"=1
2021/07/28 16:02:56 http: TLS handshake error from 127.0.0.1:43090: EOF
=== RUN   TestClusterReconcilePhases
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure/returns_no_error_if_infrastructure_ref_is_nil
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure/returns_error_if_unable_to_reconcile_infrastructure_ref
I0728 16:02:56.853639   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test-cluster" "namespace"="test-namespace" "err"="cannot create a metadata client without a rest config"
I0728 16:02:56.856765   10540 cluster_controller_phases.go:76]  "msg"="Could not find external object for cluster, requeuing" "cluster"="test-cluster" "namespace"="test-namespace" "refGroupVersionKind"={"Group":"infrastructure.cluster.x-k8s.io","Version":"v1alpha3","Kind":"InfrastructureMachine"} "refName"="test"
... skipping 105 lines ...
    --- PASS: TestReconcileBootstrap/existing_machine,_machineset_owner_and_version_v1alpha2,_and_ownerref_updated (0.00s)
=== RUN   TestReconcileInfrastructure
=== RUN   TestReconcileInfrastructure/new_machine,_infrastructure_config_ready
I0728 16:02:56.890188   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
=== RUN   TestReconcileInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machine_is_running,_infra_object_is_deleted,_expect_failed
I0728 16:02:56.891726   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
E0728 16:02:56.892008   10540 machine_controller_phases.go:246]  "msg"="Machine infrastructure reference has been deleted after being ready, setting failure state" "error"="could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine \"infra-config1\" for Machine \"machine-test\" in namespace \"default\", requeuing: requeue in 1s"  
=== RUN   TestReconcileInfrastructure/infrastructure_ref_is_paused
I0728 16:02:56.896464   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
--- PASS: TestReconcileInfrastructure (0.01s)
    --- PASS: TestReconcileInfrastructure/new_machine,_infrastructure_config_ready (0.00s)
    --- PASS: TestReconcileInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machine_is_running,_infra_object_is_deleted,_expect_failed (0.00s)
    --- PASS: TestReconcileInfrastructure/infrastructure_ref_is_paused (0.01s)
=== RUN   TestWatches
I0728 16:02:57.115027   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machine" "source"={}
E0728 16:02:57.117357   10540 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="node-1"
I0728 16:02:57.123669   10540 tracker.go:55] controllers/Machine "msg"="Adding watcher on external object" "machine"="machine-created-c8rwf" "namespace"="test-machine-watches-nfb84" "GroupVersionKind"="bootstrap.cluster.x-k8s.io/v1alpha3, Kind=BootstrapMachine"
I0728 16:02:57.123764   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machine" "source"={"Type":{"apiVersion":"bootstrap.cluster.x-k8s.io/v1alpha3","kind":"BootstrapMachine"}}
I0728 16:02:57.232304   10540 tracker.go:55] controllers/Machine "msg"="Adding watcher on external object" "machine"="machine-created-c8rwf" "namespace"="test-machine-watches-nfb84" "GroupVersionKind"="infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine"
I0728 16:02:57.232410   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machine" "source"={"Type":{"apiVersion":"infrastructure.cluster.x-k8s.io/v1alpha3","kind":"InfrastructureMachine"}}
I0728 16:02:57.333047   10540 machine_controller_noderef.go:86] controllers/Machine "msg"="Set Machine's NodeRef" "machine"="machine-created-c8rwf" "namespace"="test-machine-watches-nfb84" "noderef"="node-1"
E0728 16:02:57.342561   10540 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="node-1"
E0728 16:02:57.342620   10540 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="node-1"
E0728 16:02:57.414502   10540 machine_controller_phases.go:246] controllers/Machine "msg"="Machine infrastructure reference has been deleted after being ready, setting failure state" "error"="could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine \"infra-config1\" for Machine \"machine-created-c8rwf\" in namespace \"test-machine-watches-nfb84\", requeuing: requeue in 1s"  
E0728 16:02:57.427427   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine \"infra-config1\" for Machine \"machine-created-c8rwf\" in namespace \"test-machine-watches-nfb84\", requeuing: requeue in 1s" "controller"="machine" "name"="machine-created-c8rwf" "namespace"="test-machine-watches-nfb84"
--- PASS: TestWatches (0.62s)
=== RUN   TestIndexMachineByNodeName
=== RUN   TestIndexMachineByNodeName/when_the_machine_has_no_NodeRef
=== RUN   TestIndexMachineByNodeName/when_the_machine_has_valid_a_NodeRef
--- PASS: TestIndexMachineByNodeName (0.00s)
    --- PASS: TestIndexMachineByNodeName/when_the_machine_has_no_NodeRef (0.00s)
... skipping 2 lines ...
=== RUN   TestMachine_Reconcile/reconcile_create
I0728 16:02:57.571683   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="machine-reconcile-jvb8q" "namespace"="test-machine-watches-nfb84" "count"=1
I0728 16:02:57.571745   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="machine-reconcile-jvb8q" "namespace"="test-machine-watches-nfb84" "descendants"="Worker machines: machine-created-c8rwf" "indirect descendants count"=0
I0728 16:02:57.594997   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="machine-reconcile-jvb8q" "namespace"="test-machine-watches-nfb84" "count"=1
I0728 16:02:57.595064   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="machine-reconcile-jvb8q" "namespace"="test-machine-watches-nfb84" "descendants"="Worker machines: machine-created-c8rwf" "indirect descendants count"=0
I0728 16:02:58.427965   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-jvb8q" "machine"="machine-created-c8rwf" "namespace"="test-machine-watches-nfb84" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"node-1","uid":"907e24ca-72d0-4bbf-a114-56f0d6fb0e83","apiVersion":"v1"}
E0728 16:02:58.485327   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"machine-created-c8rwf\" not found" "controller"="machine" "name"="machine-created-c8rwf" "namespace"="test-machine-watches-nfb84"
E0728 16:02:59.527071   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="[retrieved empty dataSecretName from bootstrap provider for Machine \"machine-created-hg8c8\" in namespace \"default\", error creating client and cache for remote cluster: error fetching REST client config for remote cluster \"default/machine-reconcile-6pnbz\": failed to retrieve kubeconfig secret for Cluster default/machine-reconcile-6pnbz: secrets \"machine-reconcile-6pnbz-kubeconfig\" not found]" "controller"="machine" "name"="machine-created-hg8c8" "namespace"="default"
I0728 16:03:00.527613   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-6pnbz" "machine"="machine-created-hg8c8" "namespace"="default" "cause"="noderef is nil" "node"=null
I0728 16:03:00.558842   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-6pnbz" "machine"="machine-created-hg8c8" "namespace"="default" "cause"="noderef is nil" "node"=null
I0728 16:03:00.582197   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-6pnbz" "machine"="machine-created-hg8c8" "namespace"="default" "cause"="noderef is nil" "node"=null
E0728 16:03:00.637363   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"machine-created-hg8c8\" not found" "controller"="machine" "name"="machine-created-hg8c8" "namespace"="default"
--- PASS: TestMachine_Reconcile (3.13s)
    --- PASS: TestMachine_Reconcile/reconcile_create (3.13s)
=== RUN   TestMachineFinalizer
=== RUN   TestMachineFinalizer/should_add_a_machine_finalizer_to_the_machine_if_it_doesn't_have_one
=== RUN   TestMachineFinalizer/should_append_the_machine_finalizer_to_the_machine_if_it_already_has_a_finalizer
E0728 16:03:00.662055   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"machine-reconcile-6pnbz\" not found" "controller"="cluster" "name"="machine-reconcile-6pnbz" "namespace"="default"
--- PASS: TestMachineFinalizer (0.02s)
    --- PASS: TestMachineFinalizer/should_add_a_machine_finalizer_to_the_machine_if_it_doesn't_have_one (0.01s)
    --- PASS: TestMachineFinalizer/should_append_the_machine_finalizer_to_the_machine_if_it_already_has_a_finalizer (0.00s)
=== RUN   TestMachineOwnerReference
=== RUN   TestMachineOwnerReference/should_add_owner_reference_to_machine_referencing_a_cluster_with_correct_type_meta
I0728 16:03:00.664082   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine2" "namespace"="default" "err"="cannot create a metadata client without a rest config"
... skipping 132 lines ...
    --- PASS: TestMachineDeploymentSyncStatus/machine_set_failed (0.00s)
=== RUN   TestMachineHealthCheck_Reconcile
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_no_existing_labels_exist
I0728 16:03:00.811752   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-k766b" "namespace"="test-mhc-8s8h8" 
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_the_label_has_the_wrong_value
I0728 16:03:00.838722   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
E0728 16:03:00.838795   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-9st9m\" not found"  "cluster"="test-mhc-8s8h8/test-cluster-9st9m"
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_other_labels_are_present
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_no_existing_ones_exist
E0728 16:03:00.944233   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-k766b\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-k766b\" not found]" "controller"="machinehealthcheck" "name"="test-mhc-k766b" "namespace"="test-mhc-8s8h8"
error cannot retrieve mhc in ctx: MachineHealthCheck.cluster.x-k8s.io "test-mhc-j5dqh" not foundI0728 16:03:01.944491   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-wx88x" "namespace"="test-mhc-6trqp" 
I0728 16:03:01.944607   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-dctmb" "namespace"="test-mhc-mxpjr" 
I0728 16:03:01.944635   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-k766b" "namespace"="test-mhc-8s8h8" 
I0728 16:03:01.944662   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j5dqh" "namespace"="test-mhc-b7wk8" 
I0728 16:03:01.960117   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0728 16:03:01.981694   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j5dqh" "namespace"="test-mhc-b7wk8" 
I0728 16:03:02.042629   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j5dqh" "namespace"="test-mhc-b7wk8" 
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_modifying_existing_ones
E0728 16:03:02.069896   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-b7wk8/test-cluster-j2bh4"
E0728 16:03:02.071404   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-j2bh4\" not found" "controller"="cluster" "name"="test-cluster-j2bh4" "namespace"="test-mhc-b7wk8"
I0728 16:03:02.072610   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-fs6f6" "namespace"="test-mhc-d2z8f" 
I0728 16:03:02.085591   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0728 16:03:02.101837   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-fs6f6" "namespace"="test-mhc-d2z8f" 
I0728 16:03:02.109091   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-fs6f6" "namespace"="test-mhc-d2z8f" 
E0728 16:03:02.177920   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-d2z8f/test-cluster-jj79v"
I0728 16:03:02.181981   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-fs6f6" "namespace"="test-mhc-d2z8f" 
=== RUN   TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_all_Machines_are_healthy
I0728 16:03:02.316129   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
inframachine created: test-mhc-machine-infra-s4zg4
machine created: test-mhc-machine-28zg2
I0728 16:03:02.350594   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0728 16:03:02.451122   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-k7w5d/test-mhc-qfjjn/test-mhc-machine-28zg2/"
E0728 16:03:02.453207   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-ffw74, got []"  "node"="test-mhc-node-ffw74"
node created: test-mhc-node-ffw74
E0728 16:03:02.453499   10540 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="test-mhc-node-ffw74"
inframachine created: test-mhc-machine-infra-24kz4
machine created: test-mhc-machine-cqhjx
E0728 16:03:02.585013   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-jzn7h, got []"  "node"="test-mhc-node-jzn7h"
node created: test-mhc-node-jzn7h
E0728 16:03:02.585665   10540 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="test-mhc-node-jzn7h"
I0728 16:03:02.719755   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
I0728 16:03:02.787075   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
Cleaning up nodes, machines and infra machines.
I0728 16:03:02.799546   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
I0728 16:03:02.801391   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
I0728 16:03:02.801838   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-k7w5d/test-mhc-qfjjn/test-mhc-machine-28zg2/"
I0728 16:03:02.827350   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
I0728 16:03:02.827846   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-k7w5d/test-mhc-qfjjn/test-mhc-machine-28zg2/"
I0728 16:03:02.828300   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-k7w5d/test-mhc-qfjjn/test-mhc-machine-cqhjx/"
E0728 16:03:02.856029   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-k7w5d/test-cluster-78plz"
I0728 16:03:02.856325   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
E0728 16:03:02.856419   10540 machinehealthcheck_controller.go:128] controllers/MachineHealthCheck "msg"="Failed to fetch Cluster for MachineHealthCheck" "error"="Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "cluster"="test-cluster-78plz" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
E0728 16:03:02.856470   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machinehealthcheck" "name"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d"
=== RUN   TestMachineHealthCheck_Reconcile/it_marks_unhealthy_machines_for_remediation_when_there_is_one_unhealthy_Machine
inframachine created: test-mhc-machine-infra-kfbg4
machine created: test-mhc-machine-drz8k
E0728 16:03:02.958643   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": cannot find node with matching ProviderID" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
E0728 16:03:03.093949   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-machine-watches-nfb84/machine-reconcile-jvb8q"
I0728 16:03:03.856673   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qfjjn" "namespace"="test-mhc-k7w5d" 
I0728 16:03:03.856790   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.876279   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0728 16:03:03.905243   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.911808   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.915617   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 17 lines ...
I0728 16:03:03.949055   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.951151   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.952804   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.954447   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.956114   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.957790   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
E0728 16:03:03.958911   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:03.959419   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.961076   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.963214   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.964866   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.966533   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:03.968188   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 459 lines ...
I0728 16:03:04.962696   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:04.965336   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:04.967169   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:04.969347   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:04.970475   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:04.972444   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:04.972687   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-drz8k/"
E0728 16:03:04.973767   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:05.016564   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.016963   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-drz8k/"
node created: test-mhc-node-767tq
E0728 16:03:05.066080   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-767tq, got []"  "node"="test-mhc-node-767tq"
I0728 16:03:05.072072   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
inframachine created: test-mhc-machine-infra-xf8hn
machine created: test-mhc-machine-hk7kz
I0728 16:03:05.091792   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.097006   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.098428   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 310 lines ...
I0728 16:03:05.958709   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.961324   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.963502   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.966557   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.970582   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.972932   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
E0728 16:03:05.974106   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:05.974978   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.979170   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.981273   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.983326   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.985392   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:05.987675   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 359 lines ...
I0728 16:03:07.022472   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.027585   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.029871   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.032078   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.033198   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.036842   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
E0728 16:03:07.037633   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:07.053051   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.054257   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.055721   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.056899   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.058213   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:07.060328   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 339 lines ...
I0728 16:03:08.036466   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.045736   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.049379   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.050465   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.051441   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.053680   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.054700   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-hk7kz/"
E0728 16:03:08.054762   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:08.065560   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.066252   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-hk7kz/"
node created: test-mhc-node-gjvgp
E0728 16:03:08.111587   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-gjvgp, got []"  "node"="test-mhc-node-gjvgp"
I0728 16:03:08.143215   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
inframachine created: test-mhc-machine-infra-kk24c
machine created: test-mhc-machine-qv6ht
I0728 16:03:08.167899   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.174574   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:08.176041   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 220 lines ...
I0728 16:03:09.052594   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.055559   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.058115   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.061511   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.063967   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.066718   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
E0728 16:03:09.068999   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:09.069218   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.072484   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.074938   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.078771   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.081002   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:09.093826   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 276 lines ...
I0728 16:03:10.084392   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.085942   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.087390   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.090333   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.092598   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.099567   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
E0728 16:03:10.104142   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:10.104530   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.105970   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.107286   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.109067   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.110455   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:10.112232   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
... skipping 252 lines ...
I0728 16:03:11.071845   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.074483   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.077726   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.110546   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.114431   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.119631   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.120098   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-qv6ht/"
E0728 16:03:11.130044   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:11.137904   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.138459   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-qv6ht/"
E0728 16:03:11.208841   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-q747b, got []"  "node"="test-mhc-node-q747b"
node created: test-mhc-node-q747b
I0728 16:03:11.220773   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.221313   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-qv6ht/test-mhc-node-q747b"
Cleaning up nodes, machines and infra machines.
I0728 16:03:11.238469   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-xhkds" "namespace"="test-mhc-jbkcn" 
I0728 16:03:11.239039   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-jbkcn/test-mhc-xhkds/test-mhc-machine-qv6ht/"
Cleaning up nodes, machines and infra machines.
I0728 16:03:11.295913   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-6ph9r" "namespace"="test-mhc-jbkcn" "count"=2
I0728 16:03:11.296003   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-6ph9r" "namespace"="test-mhc-jbkcn" "descendants"="Worker machines: test-mhc-machine-drz8k,test-mhc-machine-hk7kz,test-mhc-machine-qv6ht" "indirect descendants count"=1
I0728 16:03:11.301834   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-6ph9r" "namespace"="test-mhc-jbkcn" "count"=2
I0728 16:03:11.301903   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-6ph9r" "namespace"="test-mhc-jbkcn" "descendants"="Worker machines: test-mhc-machine-drz8k,test-mhc-machine-hk7kz,test-mhc-machine-qv6ht" "indirect descendants count"=1
=== RUN   TestMachineHealthCheck_Reconcile/it_marks_unhealthy_machines_for_remediation_when_the_unhealthy_Machines_exceed_MaxUnhealthy
... skipping 7 lines ...
I0728 16:03:11.510783   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:11.551854   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:11.632969   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:11.794091   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:12.115116   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:12.130552   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-6ph9r" "machine"="test-mhc-machine-hk7kz" "namespace"="test-mhc-jbkcn" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-gjvgp"}
E0728 16:03:12.175719   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-hk7kz\" not found" "controller"="machine" "name"="test-mhc-machine-hk7kz" "namespace"="test-mhc-jbkcn"
I0728 16:03:12.757293   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:13.176114   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:14.039001   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:14.176650   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-6ph9r" "machine"="test-mhc-machine-qv6ht" "namespace"="test-mhc-jbkcn" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-q747b"}
E0728 16:03:14.215782   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-qv6ht\" not found" "controller"="machine" "name"="test-mhc-machine-qv6ht" "namespace"="test-mhc-jbkcn"
E0728 16:03:15.216479   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:16.217048   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-6ph9r" "machine"="test-mhc-machine-drz8k" "namespace"="test-mhc-jbkcn" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-767tq"}
E0728 16:03:16.265323   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-drz8k\" not found" "controller"="machine" "name"="test-mhc-machine-drz8k" "namespace"="test-mhc-jbkcn"
E0728 16:03:16.321944   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-jbkcn/test-cluster-6ph9r"
I0728 16:03:16.600108   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.273717   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:17.278012   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:17.278329   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:17.290353   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-kxd2q, got []"  "node"="test-mhc-node-kxd2q"
node created: test-mhc-node-kxd2q
I0728 16:03:17.301657   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
inframachine created: test-mhc-machine-infra-zlz4g
machine created: test-mhc-machine-bnf8r
I0728 16:03:17.324528   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.337935   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.351917   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.362047   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.403374   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.485418   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.648558   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:17.969840   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:18.278447   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:18.611133   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.291559   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.301271   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.311328   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.317854   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.325843   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:19.331395   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:19.331734   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.350728   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:19.436879   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-gg54v, got []"  "node"="test-mhc-node-gg54v"
node created: test-mhc-node-gg54v
I0728 16:03:19.451640   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
inframachine created: test-mhc-machine-infra-6748b
I0728 16:03:19.465393   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.471100   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
machine created: test-mhc-machine-rhm6f
I0728 16:03:19.484822   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.494170   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:19.892583   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:20.331789   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:21.342340   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:21.379622   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:21.389212   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:21.399508   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:21.409264   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:21.418432   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:21.426728   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
E0728 16:03:21.428717   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:21.447739   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
node created: test-mhc-node-vb9gr
E0728 16:03:21.512656   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-vb9gr, got []"  "node"="test-mhc-node-vb9gr"
I0728 16:03:21.520445   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
Cleaning up nodes, machines and infra machines.
I0728 16:03:21.544897   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
Cleaning up nodes, machines and infra machines.
I0728 16:03:21.596069   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-lmzz8" "namespace"="test-mhc-xmpjq" "count"=2
I0728 16:03:21.596156   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-lmzz8" "namespace"="test-mhc-xmpjq" "descendants"="Worker machines: test-mhc-machine-tx7kv,test-mhc-machine-bnf8r,test-mhc-machine-rhm6f" "indirect descendants count"=1
... skipping 3 lines ...
I0728 16:03:21.683996   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-7kvfh" "namespace"="test-mhc-xmpjq" 
I0728 16:03:21.722007   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
inframachine created: test-mhc-machine-infra-zsd6m
machine created: test-mhc-machine-2ngwk
I0728 16:03:21.743598   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0728 16:03:21.859994   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
E0728 16:03:22.429189   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:23.429777   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-lmzz8" "machine"="test-mhc-machine-tx7kv" "namespace"="test-mhc-xmpjq" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-kxd2q"}
E0728 16:03:23.481469   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-tx7kv\" not found" "controller"="machine" "name"="test-mhc-machine-tx7kv" "namespace"="test-mhc-xmpjq"
I0728 16:03:24.482199   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-lmzz8" "machine"="test-mhc-machine-bnf8r" "namespace"="test-mhc-xmpjq" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-gg54v"}
E0728 16:03:24.527173   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-bnf8r\" not found" "controller"="machine" "name"="test-mhc-machine-bnf8r" "namespace"="test-mhc-xmpjq"
I0728 16:03:25.527680   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-lmzz8" "machine"="test-mhc-machine-rhm6f" "namespace"="test-mhc-xmpjq" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-vb9gr"}
E0728 16:03:25.566480   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-rhm6f\" not found" "controller"="machine" "name"="test-mhc-machine-rhm6f" "namespace"="test-mhc-xmpjq"
I0728 16:03:26.576277   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:26.582528   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
E0728 16:03:26.583432   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
E0728 16:03:26.613480   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-xmpjq/test-cluster-lmzz8"
node created: test-mhc-node-z7hps
E0728 16:03:26.671747   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-z7hps, got []"  "node"="test-mhc-node-z7hps"
I0728 16:03:26.707269   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
inframachine created: test-mhc-machine-infra-qb9qs
I0728 16:03:26.722220   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:26.727250   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
machine created: test-mhc-machine-5bhb8
I0728 16:03:26.732881   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
E0728 16:03:27.583894   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:28.615767   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:28.654273   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:28.666302   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:28.672619   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:28.678340   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
E0728 16:03:28.684038   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:28.684321   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
node created: test-mhc-node-2tkrg
E0728 16:03:28.736731   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-2tkrg, got []"  "node"="test-mhc-node-2tkrg"
I0728 16:03:28.742592   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
inframachine created: test-mhc-machine-infra-cbpxl
machine created: test-mhc-machine-cjvkh
I0728 16:03:28.778407   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:28.783491   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:29.693052   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
... skipping 11 lines ...
Cleaning up nodes, machines and infra machines.
I0728 16:03:29.779341   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:29.782881   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
Cleaning up nodes, machines and infra machines.
I0728 16:03:29.787988   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-28swb" "machine"="test-mhc-machine-cjvkh" "namespace"="test-mhc-mfp8w" "cause"="noderef is nil" "node"=null
I0728 16:03:29.795356   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:29.795885   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-mfp8w/test-mhc-sfln5/test-mhc-machine-2ngwk/"
I0728 16:03:29.812485   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:29.813101   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-mfp8w/test-mhc-sfln5/test-mhc-machine-2ngwk/"
I0728 16:03:29.813570   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-mfp8w/test-mhc-sfln5/test-mhc-machine-5bhb8/"
I0728 16:03:29.823781   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-28swb" "namespace"="test-mhc-mfp8w" "count"=2
I0728 16:03:29.823860   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-28swb" "namespace"="test-mhc-mfp8w" "descendants"="Worker machines: test-mhc-machine-2ngwk,test-mhc-machine-5bhb8" "indirect descendants count"=0
E0728 16:03:29.825446   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-cjvkh\" not found" "controller"="machine" "name"="test-mhc-machine-cjvkh" "namespace"="test-mhc-mfp8w"
I0728 16:03:29.829683   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-28swb" "namespace"="test-mhc-mfp8w" "count"=2
I0728 16:03:29.829723   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-28swb" "namespace"="test-mhc-mfp8w" "descendants"="Worker machines: test-mhc-machine-2ngwk,test-mhc-machine-5bhb8" "indirect descendants count"=0
I0728 16:03:29.830970   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:29.831312   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-mfp8w/test-mhc-sfln5/test-mhc-machine-2ngwk/"
I0728 16:03:29.831627   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-mfp8w/test-mhc-sfln5/test-mhc-machine-5bhb8/"
=== RUN   TestMachineHealthCheck_Reconcile/when_a_Machine_has_no_Node_ref_for_longer_than_the_NodeStartupTimeout
=== RUN   TestMachineHealthCheck_Reconcile/when_a_Machine's_Node_has_gone_away
=== RUN   TestMachineHealthCheck_Reconcile/should_react_when_a_Node_transitions_to_unhealthy
E0728 16:03:29.833753   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machinehealthchecks.cluster.x-k8s.io \"test-mhc-sfln5\" not found" "controller"="machinehealthcheck" "name"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w"
inframachine created: test-mhc-machine-infra-wfvpw
machine created: test-mhc-machine-wr8dx
I0728 16:03:30.826008   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-28swb" "machine"="test-mhc-machine-2ngwk" "namespace"="test-mhc-mfp8w" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-z7hps"}
I0728 16:03:30.834074   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sfln5" "namespace"="test-mhc-mfp8w" 
I0728 16:03:30.834144   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:30.844718   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
E0728 16:03:30.863405   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-2ngwk\" not found" "controller"="machine" "name"="test-mhc-machine-2ngwk" "namespace"="test-mhc-mfp8w"
I0728 16:03:30.963566   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:30.964655   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:30.966385   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:30.968133   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:30.969929   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:30.971580   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
... skipping 524 lines ...
I0728 16:03:31.888677   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.891497   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.893371   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.895221   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.897019   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.899271   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
E0728 16:03:31.900367   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-5bhb8\" not found" "controller"="machine" "name"="test-mhc-machine-5bhb8" "namespace"="test-mhc-mfp8w"
I0728 16:03:31.901120   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.902722   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.904551   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.906291   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.908038   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:31.909829   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
... skipping 551 lines ...
I0728 16:03:32.902831   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.904429   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.906136   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.907103   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.907860   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.909886   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.910801   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-bdr4z/test-mhc-jg8fq/test-mhc-machine-wr8dx/"
E0728 16:03:32.912131   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:32.919493   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.919678   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-bdr4z/test-mhc-jg8fq/test-mhc-machine-wr8dx/"
E0728 16:03:32.965931   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-nl6xf, got []"  "node"="test-mhc-node-nl6xf"
E0728 16:03:32.965945   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-nl6xf, got []"  "node"="test-mhc-node-nl6xf"
node created: test-mhc-node-nl6xf
I0728 16:03:32.971643   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.982879   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.996065   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:32.996476   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-bdr4z/test-mhc-jg8fq/test-mhc-machine-wr8dx/test-mhc-node-nl6xf"
I0728 16:03:33.009093   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:33.009533   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-bdr4z/test-mhc-jg8fq/test-mhc-machine-wr8dx/test-mhc-node-nl6xf"
I0728 16:03:33.013320   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:33.013619   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-bdr4z/test-mhc-jg8fq/test-mhc-machine-wr8dx/test-mhc-node-nl6xf"
Cleaning up nodes, machines and infra machines.
I0728 16:03:33.020072   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:33.020444   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-bdr4z/test-mhc-jg8fq/test-mhc-machine-wr8dx/"
I0728 16:03:33.034150   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-wn28q" "namespace"="test-mhc-bdr4z" "descendants"="Worker machines: test-mhc-machine-wr8dx" "indirect descendants count"=1
=== RUN   TestMachineHealthCheck_Reconcile/when_in_a_MachineSet,_unhealthy_machines_should_be_deleted
I0728 16:03:33.038664   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-wn28q" "namespace"="test-mhc-bdr4z" "descendants"="Worker machines: test-mhc-machine-wr8dx" "indirect descendants count"=1
I0728 16:03:33.187310   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jg8fq" "namespace"="test-mhc-bdr4z" 
I0728 16:03:33.189458   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" "creating"=1 "need"=1
I0728 16:03:33.189491   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(1) \u003e currentMachineCount(0) )" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" 
I0728 16:03:33.197683   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"mhc-ms-7xpf4-6n6pt\"" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" 
I0728 16:03:33.283641   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:33.295497   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0728 16:03:33.307385   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:33.310344   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
E0728 16:03:33.912516   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:34.307566   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
E0728 16:03:34.840869   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-mfp8w/test-cluster-28swb"
I0728 16:03:34.913037   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-wn28q" "machine"="test-mhc-machine-wr8dx" "namespace"="test-mhc-bdr4z" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-nl6xf"}
E0728 16:03:34.947310   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-wr8dx\" not found" "controller"="machine" "name"="test-mhc-machine-wr8dx" "namespace"="test-mhc-bdr4z"
I0728 16:03:35.308711   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.953998   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.954101   10540 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.957643   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.957745   10540 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.966205   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
... skipping 4 lines ...
I0728 16:03:35.973715   10540 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.977106   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.977136   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.983792   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:35.983831   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.309866   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.310321   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1s" "reason"="NodeStartupTimeout" "target"="test-mhc-nv8gr/test-mhc-b68rp/mhc-ms-7xpf4-6n6pt/"
I0728 16:03:36.316536   10540 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.319991   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.320034   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.320305   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.320673   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1s" "reason"="NodeStartupTimeout" "target"="test-mhc-nv8gr/test-mhc-b68rp/mhc-ms-7xpf4-6n6pt/"
I0728 16:03:36.326165   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.326273   10540 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.326505   10540 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1s" "reason"="NodeStartupTimeout" "target"="test-mhc-nv8gr/test-mhc-b68rp/mhc-ms-7xpf4-6n6pt/"
I0728 16:03:36.335857   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.335895   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" 
I0728 16:03:36.413005   10540 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-rf56h" "namespace"="test-mhc-nv8gr" "count"=1
I0728 16:03:36.413056   10540 cluster_controller.go:241] controllers/Cluster "msg"="Deleting child" "cluster"="test-cluster-rf56h" "namespace"="test-mhc-nv8gr" "gvk"="cluster.x-k8s.io/v1alpha3, Kind=MachineSet" "name"="mhc-ms-7xpf4"
I0728 16:03:36.413640   10540 machineset_controller.go:237] controllers/MachineSet "msg"="Deleting unhealthy machine" "machineset"="mhc-ms-7xpf4" "namespace"="test-mhc-nv8gr" "machine"="mhc-ms-7xpf4-6n6pt"
I0728 16:03:36.417044   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-b68rp" "namespace"="test-mhc-nv8gr" 
... skipping 4 lines ...
=== RUN   TestMachineHealthCheck_Reconcile/When_remediationTemplate_is_set_and_node_transitions_to_unhealthy,_new_Remediation_Request_should_be_created
I0728 16:03:36.570156   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
inframachine created: test-mhc-machine-infra-8rw4m
I0728 16:03:36.584779   10540 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
machine created: test-mhc-machine-sbm67
I0728 16:03:36.614433   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-rf56h" "machine"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr" "cause"="cluster is being deleted" "node"=null
E0728 16:03:36.638703   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"mhc-ms-7xpf4-6n6pt\" not found" "controller"="machine" "name"="mhc-ms-7xpf4-6n6pt" "namespace"="test-mhc-nv8gr"
E0728 16:03:36.693021   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-v9ctf, got []"  "node"="test-mhc-node-v9ctf"
node created: test-mhc-node-v9ctf
E0728 16:03:36.693491   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-v9ctf, got []"  "node"="test-mhc-node-v9ctf"
E0728 16:03:36.693709   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-v9ctf, got []"  "node"="test-mhc-node-v9ctf"
I0728 16:03:36.717129   10540 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "remediation request name"="test-mhc-machine-sbm67" "target"="test-mhc-jbszg/test-mhc-2dw9l/test-mhc-machine-sbm67/"
I0728 16:03:36.737326   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
I0728 16:03:36.752566   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
I0728 16:03:36.762914   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
I0728 16:03:36.765954   10540 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "remediation request name"="test-mhc-machine-sbm67" "target"="test-mhc-jbszg/test-mhc-2dw9l/test-mhc-machine-sbm67/test-mhc-node-v9ctf"
I0728 16:03:36.779043   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
Cleaning up nodes, machines and infra machines.
I0728 16:03:36.790247   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
I0728 16:03:36.792128   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
I0728 16:03:36.795493   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-2dw9l" "namespace"="test-mhc-jbszg" 
I0728 16:03:36.803712   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-85dg7" "namespace"="test-mhc-jbszg" "descendants"="Worker machines: test-mhc-machine-sbm67" "indirect descendants count"=1
... skipping 423 lines ...
I0728 16:03:37.659601   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.661219   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.662901   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.664621   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.666427   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.667956   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
E0728 16:03:37.669425   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-sbm67\" not found" "controller"="machine" "name"="test-mhc-machine-sbm67" "namespace"="test-mhc-jbszg"
I0728 16:03:37.669671   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.671271   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.672861   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.674539   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.676113   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:37.677734   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
... skipping 208 lines ...
I0728 16:03:38.038908   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.040481   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.042125   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.043842   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.045491   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.047104   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
E0728 16:03:38.048487   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-bdr4z/test-cluster-wn28q"
I0728 16:03:38.048684   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.050203   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.051719   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.053455   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.055029   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.056642   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
... skipping 362 lines ...
I0728 16:03:38.672732   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.674391   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.676059   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.676718   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.677828   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.679901   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
E0728 16:03:38.682352   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
I0728 16:03:38.685181   10540 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "remediation request name"="test-mhc-machine-zsxk7" "target"="test-mhc-s88nc/test-mhc-thvjm/test-mhc-machine-zsxk7/"
I0728 16:03:38.694410   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
E0728 16:03:38.747159   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-dz2jv, got []"  "node"="test-mhc-node-dz2jv"
node created: test-mhc-node-dz2jv
E0728 16:03:38.747436   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-dz2jv, got []"  "node"="test-mhc-node-dz2jv"
E0728 16:03:38.747559   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-dz2jv, got []"  "node"="test-mhc-node-dz2jv"
I0728 16:03:38.754342   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.774169   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.779632   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.788946   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.794095   10540 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "remediation request name"="test-mhc-machine-zsxk7" "target"="test-mhc-s88nc/test-mhc-thvjm/test-mhc-machine-zsxk7/test-mhc-node-dz2jv"
I0728 16:03:38.810058   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.812655   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.823674   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.969007   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.971151   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
Cleaning up nodes, machines and infra machines.
I0728 16:03:38.985515   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:38.991132   10540 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="" "reason"="NodeNotFound" "remediation request name"="test-mhc-machine-zsxk7" "target"="test-mhc-s88nc/test-mhc-thvjm/test-mhc-machine-zsxk7/"
I0728 16:03:38.995818   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-4sxnj" "namespace"="test-mhc-s88nc" "descendants"="Worker machines: test-mhc-machine-zsxk7" "indirect descendants count"=1
I0728 16:03:39.000718   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-4sxnj" "namespace"="test-mhc-s88nc" "descendants"="Worker machines: test-mhc-machine-zsxk7" "indirect descendants count"=1
--- PASS: TestMachineHealthCheck_Reconcile (38.21s)
    --- PASS: TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_no_existing_labels_exist (0.04s)
    --- PASS: TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_the_label_has_the_wrong_value (0.04s)
    --- PASS: TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_other_labels_are_present (0.03s)
... skipping 12 lines ...
    --- SKIP: TestMachineHealthCheck_Reconcile/when_a_machine_is_paused (0.00s)
        machinehealthcheck_controller_test.go:933: skipping until made stable
    --- PASS: TestMachineHealthCheck_Reconcile/When_remediationTemplate_is_set_and_node_transitions_to_unhealthy,_new_Remediation_Request_should_be_created (0.38s)
    --- PASS: TestMachineHealthCheck_Reconcile/When_remediationTemplate_is_set_and_node_transitions_back_to_healthy,_new_Remediation_Request_should_be_deleted (2.19s)
=== RUN   TestClusterToMachineHealthCheck
=== RUN   TestClusterToMachineHealthCheck/when_the_object_passed_isn't_a_cluster
E0728 16:03:39.005261   10540 machinehealthcheck_controller.go:416]  "msg"="expected a Cluster" "error"="incorrect type"  "type"="handler.MapObject"
=== RUN   TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Cluster_in_the_same_namespace
=== RUN   TestClusterToMachineHealthCheck/when_2_MachineHealthChecks_exists_for_the_Cluster_in_the_same_namespace
=== RUN   TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_the_same_namespace
E0728 16:03:39.005899   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machinehealthchecks.cluster.x-k8s.io \"test-mhc-thvjm\" not found" "controller"="machinehealthcheck" "name"="test-mhc-thvjm" "namespace"="test-mhc-s88nc"
=== RUN   TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_another_namespace
--- PASS: TestClusterToMachineHealthCheck (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_the_object_passed_isn't_a_cluster (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Cluster_in_the_same_namespace (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_2_MachineHealthChecks_exists_for_the_Cluster_in_the_same_namespace (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_the_same_namespace (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_another_namespace (0.00s)
=== RUN   TestMachineToMachineHealthCheck
=== RUN   TestMachineToMachineHealthCheck/when_the_object_passed_isn't_a_machine
E0728 16:03:39.006478   10540 machinehealthcheck_controller.go:445]  "msg"="expected a Machine" "error"="incorrect type"  "type"="handler.MapObject"
=== RUN   TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_the_same_namespace
=== RUN   TestMachineToMachineHealthCheck/when_2_MachineHealthChecks_match_labels_for_the_Machine_in_the_same_namespace
=== RUN   TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_does_not_match_labels_for_the_Machine_in_the_same_namespace
=== RUN   TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_another_namespace
--- PASS: TestMachineToMachineHealthCheck (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_the_object_passed_isn't_a_machine (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_the_same_namespace (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_2_MachineHealthChecks_match_labels_for_the_Machine_in_the_same_namespace (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_does_not_match_labels_for_the_Machine_in_the_same_namespace (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_another_namespace (0.00s)
=== RUN   TestNodeToMachineHealthCheck
=== RUN   TestNodeToMachineHealthCheck/when_the_object_passed_isn't_a_Node
E0728 16:03:39.007608   10540 machinehealthcheck_controller.go:474]  "msg"="expected a Node" "error"="incorrect type"  "type"="handler.MapObject"
=== RUN   TestNodeToMachineHealthCheck/when_no_Machine_exists_for_the_Node
E0728 16:03:39.007779   10540 machinehealthcheck_controller.go:480]  "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node node1, got []"  "node"="node1"
=== RUN   TestNodeToMachineHealthCheck/when_two_Machines_exist_for_the_Node
E0728 16:03:39.008071   10540 machinehealthcheck_controller.go:480]  "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node node1, got [machine1 machine2]"  "node"="node1"
=== RUN   TestNodeToMachineHealthCheck/when_no_MachineHealthCheck_exists_for_the_Node_in_the_Machine's_namespace
=== RUN   TestNodeToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Node_in_the_Machine's_namespace
=== RUN   TestNodeToMachineHealthCheck/when_two_MachineHealthChecks_exist_for_the_Node_in_the_Machine's_namespace
=== RUN   TestNodeToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Node,_but_not_in_the_Machine's_cluster
--- PASS: TestNodeToMachineHealthCheck (0.00s)
    --- PASS: TestNodeToMachineHealthCheck/when_the_object_passed_isn't_a_Node (0.00s)
... skipping 31 lines ...
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_nil (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_not_an_int_or_percentage (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_an_int (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_a_40%_(of_5) (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_a_60%_(of_7) (0.00s)
=== RUN   TestPatchTargets
I0728 16:03:39.010052   10540 machinehealthcheck_controller.go:387]  "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="" "target"="default/mhc/machine1/"
I0728 16:03:39.010457   10540 machinehealthcheck_controller.go:387]  "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="" "target"="default/mhc/machine2/"
E0728 16:03:39.011557   10540 machinehealthcheck_controller.go:320]  "msg"="failed to patch healthy machine status for machine" "error"="machines.cluster.x-k8s.io \"machine1\" not found"  "machine"="machine1"
--- PASS: TestPatchTargets (0.00s)
=== RUN   TestGetTargetsFromMHC
=== RUN   TestGetTargetsFromMHC/with_no_matching_machines
=== RUN   TestGetTargetsFromMHC/when_a_machine's_node_is_missing
=== RUN   TestGetTargetsFromMHC/when_a_machine's_labels_do_not_match_the_selector
=== RUN   TestGetTargetsFromMHC/with_multiple_machines,_should_match_correct_nodes
... skipping 26 lines ...
--- PASS: TestMachineSetOwnerReference (0.00s)
    --- PASS: TestMachineSetOwnerReference/should_add_cluster_owner_reference_to_machine_set (0.00s)
    --- PASS: TestMachineSetOwnerReference/should_not_add_cluster_owner_reference_if_machine_is_owned_by_a_machine_deployment (0.00s)
=== RUN   TestMachineSetReconcile
=== RUN   TestMachineSetReconcile/ignore_machine_sets_marked_for_deletion
=== RUN   TestMachineSetReconcile/records_event_if_reconcile_fails
E0728 16:03:39.017371   10540 machineset_controller.go:151]  "msg"="Failed to reconcile MachineSet" "error"="failed to calculate MachineSet's Status: failed to calculate status for MachineSet default/machineset1: invalid label key \"--$-invalid\": name part must consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyName',  or 'my.name',  or '123-abc', regex used for validation is '([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]')" "machineset"="machineset1" "namespace"="default" 
--- PASS: TestMachineSetReconcile (0.00s)
    --- PASS: TestMachineSetReconcile/ignore_machine_sets_marked_for_deletion (0.00s)
    --- PASS: TestMachineSetReconcile/records_event_if_reconcile_fails (0.00s)
=== RUN   TestMachineSetToMachines
=== RUN   TestMachineSetToMachines/should_return_empty_request_when_controller_is_set
=== RUN   TestMachineSetToMachines/should_return_nil_if_machine_has_no_owner_reference
... skipping 8 lines ...
--- PASS: TestAdoptOrphan (0.00s)
=== RUN   TestHasMatchingLabels
=== RUN   TestHasMatchingLabels/machine_set_and_machine_have_matching_labels
=== RUN   TestHasMatchingLabels/machine_set_and_machine_do_not_have_matching_labels
=== RUN   TestHasMatchingLabels/machine_set_has_empty_selector
=== RUN   TestHasMatchingLabels/machine_set_has_bad_selector
E0728 16:03:39.018411   10540 machineset_controller.go:569]  "msg"="Unable to convert selector" "error"="\"bad-operator\" is not a valid pod selector operator" "machine"="match" "machineset"="" "namespace"="" 
--- PASS: TestHasMatchingLabels (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_and_machine_have_matching_labels (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_and_machine_do_not_have_matching_labels (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_has_empty_selector (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_has_bad_selector (0.00s)
=== RUN   TestMachineToDelete
... skipping 73 lines ...
I0728 16:03:39.311475   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "creating"=1 "need"=3
I0728 16:03:39.311517   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(3) \u003e currentMachineCount(2) )" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" 
I0728 16:03:39.319926   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-lhzjv-6657c7fddb-th58s\"" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" 
I0728 16:03:39.427838   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "creating"=1 "need"=1
I0728 16:03:39.427936   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(1) \u003e currentMachineCount(0) )" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" 
I0728 16:03:39.436009   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-lhzjv-cdfc6fd6c-slndl\"" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" 
E0728 16:03:39.472867   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.472867   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.472872   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.576501   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.576543   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.576501   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.576568   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.577013   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.577047   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-slndl-sh6qq, got []"  "node"="md-lhzjv-cdfc6fd6c-slndl-sh6qq"
E0728 16:03:39.598049   10540 machineset_controller.go:625] controllers/MachineSet "msg"="Unable to retrieve Node status" "error"="error retrieving node md-lhzjv-cdfc6fd6c-slndl-sh6qq for machine md-test/md-lhzjv-cdfc6fd6c-slndl: the cache is not started, can not read objects" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" 
I0728 16:03:39.618581   10540 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "deleting"=1 "need"=2
I0728 16:03:39.618615   10540 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "delete-policy"="Random"
I0728 16:03:39.622476   10540 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "machine"="md-lhzjv-6657c7fddb-sb7np"
I0728 16:03:39.637851   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "creating"=1 "need"=2
I0728 16:03:39.637891   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(2) \u003e currentMachineCount(1) )" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" 
I0728 16:03:39.645200   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-lhzjv-cdfc6fd6c-n6922\"" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" 
I0728 16:03:39.682949   10540 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-4sxnj" "machine"="test-mhc-machine-zsxk7" "namespace"="test-mhc-s88nc" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-dz2jv"}
E0728 16:03:39.702944   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.702994   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.703741   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.705433   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.705487   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.705756   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.705821   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.705752   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.705866   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-n6922-qr7fj, got []"  "node"="md-lhzjv-cdfc6fd6c-n6922-qr7fj"
E0728 16:03:39.709897   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-zsxk7\" not found" "controller"="machine" "name"="test-mhc-machine-zsxk7" "namespace"="test-mhc-s88nc"
I0728 16:03:39.757137   10540 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "deleting"=1 "need"=1
I0728 16:03:39.757191   10540 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "delete-policy"="Random"
I0728 16:03:39.764152   10540 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "machine"="md-lhzjv-6657c7fddb-kmcxv"
I0728 16:03:39.778523   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "creating"=1 "need"=3
I0728 16:03:39.778578   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(3) \u003e currentMachineCount(2) )" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" 
I0728 16:03:39.790484   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-lhzjv-cdfc6fd6c-cgrzq\"" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" 
E0728 16:03:39.830572   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.830657   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.830659   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.934123   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.934188   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.934235   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.934244   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.934269   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
E0728 16:03:39.934299   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-cdfc6fd6c-cgrzq-mcjn4, got []"  "node"="md-lhzjv-cdfc6fd6c-cgrzq-mcjn4"
I0728 16:03:39.961190   10540 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "deleting"=1 "need"=0
I0728 16:03:39.961236   10540 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "delete-policy"="Random"
I0728 16:03:39.965424   10540 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-lhzjv-6657c7fddb" "namespace"="md-test" "machine"="md-lhzjv-6657c7fddb-th58s"
I0728 16:03:40.006153   10540 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-thvjm" "namespace"="test-mhc-s88nc" 
I0728 16:03:40.214832   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" "creating"=1 "need"=1
I0728 16:03:40.214870   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(1) \u003e currentMachineCount(0) )" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" 
I0728 16:03:40.223066   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-lhzjv-74d45c49c5-xmh4v\"" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" 
E0728 16:03:40.310586   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.310607   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.310679   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.312985   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.313003   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.313022   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.313048   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.313254   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
E0728 16:03:40.313294   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-xmh4v-j92dg, got []"  "node"="md-lhzjv-74d45c49c5-xmh4v-j92dg"
I0728 16:03:40.340451   10540 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "deleting"=1 "need"=2
I0728 16:03:40.340485   10540 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "delete-policy"="Random"
I0728 16:03:40.343990   10540 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "machine"="md-lhzjv-cdfc6fd6c-n6922"
I0728 16:03:40.361277   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" "creating"=1 "need"=2
I0728 16:03:40.361314   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(2) \u003e currentMachineCount(1) )" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" 
I0728 16:03:40.370418   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-lhzjv-74d45c49c5-qlflv\"" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" 
E0728 16:03:40.437407   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.437470   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.437657   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.540874   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.540958   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.540874   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.541015   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.541076   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
E0728 16:03:40.541136   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-qlflv-bfvzm, got []"  "node"="md-lhzjv-74d45c49c5-qlflv-bfvzm"
I0728 16:03:40.571044   10540 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "deleting"=1 "need"=1
I0728 16:03:40.571077   10540 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "delete-policy"="Random"
I0728 16:03:40.577808   10540 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "machine"="md-lhzjv-cdfc6fd6c-slndl"
I0728 16:03:40.685899   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" "creating"=1 "need"=3
I0728 16:03:40.685958   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(3) \u003e currentMachineCount(2) )" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" 
I0728 16:03:40.694372   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-lhzjv-74d45c49c5-l6k4d\"" "machineset"="md-lhzjv-74d45c49c5" "namespace"="md-test" 
E0728 16:03:40.710497   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
E0728 16:03:40.781479   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.781547   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.781774   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.885092   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.885125   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.885135   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.885170   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.885367   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
E0728 16:03:40.885399   10540 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-lhzjv-74d45c49c5-l6k4d-hbql7, got []"  "node"="md-lhzjv-74d45c49c5-l6k4d-hbql7"
I0728 16:03:40.906401   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-lhzjv-74d45c49c5,md-lhzjv-cdfc6fd6c;Worker machines: md-lhzjv-74d45c49c5-xmh4v,md-lhzjv-74d45c49c5-l6k4d,md-lhzjv-6657c7fddb-prj6w,md-lhzjv-6657c7fddb-4fd2h,md-lhzjv-74d45c49c5-qlflv,md-lhzjv-cdfc6fd6c-cgrzq" "indirect descendants count"=8
•I0728 16:03:40.911171   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-lhzjv-74d45c49c5,md-lhzjv-cdfc6fd6c;Worker machines: md-lhzjv-6657c7fddb-prj6w,md-lhzjv-6657c7fddb-4fd2h,md-lhzjv-74d45c49c5-qlflv,md-lhzjv-cdfc6fd6c-cgrzq,md-lhzjv-74d45c49c5-xmh4v,md-lhzjv-74d45c49c5-l6k4d" "indirect descendants count"=8
E0728 16:03:40.911694   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machinedeployments.cluster.x-k8s.io \"md-lhzjv\" not found" "controller"="machinedeployment" "name"="md-lhzjv" "namespace"="md-test"
I0728 16:03:40.913668   10540 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "deleting"=1 "need"=0
I0728 16:03:40.913695   10540 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "delete-policy"="Random"
I0728 16:03:40.920595   10540 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-lhzjv-cdfc6fd6c" "namespace"="md-test" "machine"="md-lhzjv-cdfc6fd6c-cgrzq"
•E0728 16:03:41.032293   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test1-d58gh\" not found" "controller"="cluster" "name"="test1-d58gh" "namespace"="default"
I0728 16:03:41.765276   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:41.765314   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:41.777953   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:03:41.777992   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
E0728 16:03:41.808027   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:42.055753   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-nv8gr/test-cluster-rf56h"
E0728 16:03:42.064007   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-jbszg/test-cluster-85dg7"
•I0728 16:03:42.148281   10540 conversion.go:64] controllers/Cluster "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test2-8gkhx" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
I0728 16:03:42.267462   10540 conversion.go:64] controllers/Cluster "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test2-8gkhx" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
E0728 16:03:42.271103   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to find a CustomResourceDefinition for /, Kind= with contract \"cluster.x-k8s.io/v1alpha3\"" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
E0728 16:03:42.830724   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
E0728 16:03:43.281895   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
•I0728 16:03:43.858181   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:43.858222   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:43.861415   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:03:43.861448   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
E0728 16:03:43.866463   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:44.292908   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test3-84lht\" not found" "controller"="cluster" "name"="test3-84lht" "namespace"="default"
E0728 16:03:44.870869   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
E0728 16:03:45.293593   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
E0728 16:03:45.879207   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:46.314204   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-s88nc/test-cluster-4sxnj"
E0728 16:03:46.314210   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
•I0728 16:03:46.883714   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:46.883760   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:46.887328   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:03:46.887361   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
E0728 16:03:46.891578   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
I0728 16:03:47.314801   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-lhzjv-74d45c49c5,md-lhzjv-cdfc6fd6c;Worker machines: md-lhzjv-6657c7fddb-prj6w,md-lhzjv-6657c7fddb-4fd2h,md-lhzjv-74d45c49c5-qlflv,md-lhzjv-74d45c49c5-xmh4v,md-lhzjv-74d45c49c5-l6k4d" "indirect descendants count"=7
E0728 16:03:47.318697   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
E0728 16:03:47.896213   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:48.319441   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
E0728 16:03:48.901452   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
E0728 16:03:49.328533   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
•I0728 16:03:49.905576   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:49.905632   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:49.909747   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:03:49.909790   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
E0728 16:03:49.913972   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:50.329275   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
E0728 16:03:50.914380   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-28zg2\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-28zg2" "namespace"="test-mhc-k7w5d"
E0728 16:03:51.330069   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
E0728 16:03:51.919151   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
E0728 16:03:52.340059   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
E0728 16:03:52.923820   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:53.340763   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
I0728 16:03:53.928586   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:53.928632   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:53.932018   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:03:53.932050   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
E0728 16:03:53.932481   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-78plz\" for machine \"test-mhc-machine-cqhjx\" in namespace \"test-mhc-k7w5d\": Cluster.cluster.x-k8s.io \"test-cluster-78plz\" not found" "controller"="machine" "name"="test-mhc-machine-cqhjx" "namespace"="test-mhc-k7w5d"
I0728 16:03:54.341368   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-lhzjv-74d45c49c5,md-lhzjv-cdfc6fd6c;Worker machines: md-lhzjv-6657c7fddb-prj6w,md-lhzjv-6657c7fddb-4fd2h,md-lhzjv-74d45c49c5-qlflv,md-lhzjv-74d45c49c5-xmh4v,md-lhzjv-74d45c49c5-l6k4d" "indirect descendants count"=7
E0728 16:03:54.342018   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
E0728 16:03:54.936962   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
E0728 16:03:55.342792   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
E0728 16:03:55.951631   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:56.343600   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
I0728 16:03:56.955934   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:56.955971   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:03:56.959431   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:03:56.959472   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
E0728 16:03:56.963440   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
E0728 16:03:57.344330   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
I0728 16:03:57.978138   10540 conversion.go:64] controllers/Machine "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="test6-rgqn2" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
E0728 16:03:57.992660   10540 machine_controller_noderef.go:73] controllers/Machine "msg"="Failed to retrieve Node by ProviderID" "error"="the cache is not started, can not read objects" "machine"="test6-rgqn2" "namespace"="default" 
E0728 16:03:58.007095   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="[failed to find a CustomResourceDefinition for /, Kind= with contract \"cluster.x-k8s.io/v1alpha3\", the cache is not started, can not read objects]" "controller"="machine" "name"="test6-rgqn2" "namespace"="default"
E0728 16:03:58.345110   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
E0728 16:03:59.011888   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:03:59.345892   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"
E0728 16:04:00.016491   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-qlflv\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-qlflv" "namespace"="md-test"
E0728 16:04:00.346743   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
I0728 16:04:01.021427   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:04:01.021469   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-prj6w" "namespace"="md-test" 
I0728 16:04:01.025167   10540 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:04:01.025206   10540 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-lhzjv-6657c7fddb-4fd2h" "namespace"="md-test" 
I0728 16:04:01.038830   10540 conversion.go:64] controllers/Machine "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="test6-rgqn2" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
I0728 16:04:01.039045   10540 machine_controller_noderef.go:86] controllers/Machine "msg"="Set Machine's NodeRef" "machine"="test6-rgqn2" "namespace"="default" "noderef"="id-node-1"
E0728 16:04:01.052188   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to find a CustomResourceDefinition for /, Kind= with contract \"cluster.x-k8s.io/v1alpha3\"" "controller"="machine" "name"="test6-rgqn2" "namespace"="default"
I0728 16:04:01.347389   10540 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-lhzjv-74d45c49c5,md-lhzjv-cdfc6fd6c;Worker machines: md-lhzjv-74d45c49c5-xmh4v,md-lhzjv-74d45c49c5-l6k4d,md-lhzjv-6657c7fddb-prj6w,md-lhzjv-6657c7fddb-4fd2h,md-lhzjv-74d45c49c5-qlflv" "indirect descendants count"=7
E0728 16:04:01.347877   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-lh64q: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-lh64q" "namespace"="default"
E0728 16:04:02.056996   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-lhzjv-74d45c49c5-xmh4v\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-lhzjv-74d45c49c5-xmh4v" "namespace"="md-test"
E0728 16:04:02.352410   10540 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-8gkhx: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-8gkhx" "namespace"="default"

------------------------------
• [SLOW TEST:13.058 seconds]
Cluster Reconciler
/home/prow/go/src/sigs.k8s.io/cluster-api/controllers/cluster_controller_test.go:45
  Should successfully set Status.ControlPlaneInitialized on the cluster object if controlplane is ready
... skipping 21 lines ...
I0728 16:04:02.462957   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
•I0728 16:04:02.464319   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
I0728 16:04:02.464950   10540 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
I0728 16:04:02.465490   10540 machine_controller_phases.go:278]  "msg"="Infrastructure provider is not ready, requeuing" "machine"="machine-test" "namespace"="default" 
I0728 16:04:02.465518   10540 machine_controller_noderef.go:45]  "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="machine-test" "namespace"="default" 
•I0728 16:04:02.465982   10540 machine_controller.go:327]  "msg"="Draining node" "cluster"="test-cluster" "machine"="machine-test" "namespace"="default" "node"="machine-test-node"
E0728 16:04:02.467106   10540 machine_controller.go:494]  "msg"="Error creating a remote client while deleting Machine, won't retry" "error"="failed to create REST configuration for Cluster default/test-cluster: invalid configuration: no configuration has been provided" "cluster"="test-cluster" "machine"="machine-test" "namespace"="default" "node"="machine-test-node" 
•I0728 16:04:02.529746   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="ms-wj7bw" "namespace"="ms-test" "creating"=2 "need"=2
I0728 16:04:02.529785   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 2, ( spec.replicas(2) \u003e currentMachineCount(0) )" "machineset"="ms-wj7bw" "namespace"="ms-test" 
I0728 16:04:02.540442   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 2 with name \"ms-wj7bw-wmqzk\"" "machineset"="ms-wj7bw" "namespace"="ms-test" 
I0728 16:04:02.540484   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 2 of 2, ( spec.replicas(2) \u003e currentMachineCount(0) )" "machineset"="ms-wj7bw" "namespace"="ms-test" 
I0728 16:04:02.551128   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 2 of 2 with name \"ms-wj7bw-pkr86\"" "machineset"="ms-wj7bw" "namespace"="ms-test" 
I0728 16:04:02.664823   10540 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="ms-wj7bw" "namespace"="ms-test" "creating"=1 "need"=2
I0728 16:04:02.664862   10540 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(2) \u003e currentMachineCount(1) )" "machineset"="ms-wj7bw" "namespace"="ms-test" 
I0728 16:04:02.676906   10540 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"ms-wj7bw-z9jw6\"" "machineset"="ms-wj7bw" "namespace"="ms-test" 
E0728 16:04:02.792775   10540 machineset_controller.go:625] controllers/MachineSet "msg"="Unable to retrieve Node status" "error"="error retrieving node ms-wj7bw-z9jw6-mkrt9 for machine ms-test/ms-wj7bw-z9jw6: the cache is not started, can not read objects" "machineset"="ms-wj7bw" "namespace"="ms-test" 
E0728 16:04:02.919668   10540 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="ms-test/test-cluster"
•

Ran 16 of 16 Specs in 23.903 seconds
SUCCESS! -- 16 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestGinkgoSuite (23.90s)
PASS
Tearing down test suite
I0728 16:04:02.922181   10540 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machinehealthcheck" 
I0728 16:04:02.922220   10540 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machinedeployment" 
I0728 16:04:02.922254   10540 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  
I0728 16:04:02.922280   10540 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machineset" 
I0728 16:04:02.922312   10540 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machine" 
I0728 16:04:02.922334   10540 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="cluster" 
I0728 16:04:02.922352   10540 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="cluster" 
E0728 16:04:02.959479   10540 reflector.go:309] pkg/mod/k8s.io/client-go@v0.17.9/tools/cache/reflector.go:105: Failed to watch *v1.Node: Get http://127.0.0.1:42849/api/v1/nodes?allowWatchBookmarks=true&resourceVersion=1217&timeout=10s&timeoutSeconds=522&watch=true: dial tcp 127.0.0.1:42849: connect: connection refused
E0728 16:04:02.959484   10540 reflector.go:309] pkg/mod/k8s.io/client-go@v0.17.9/tools/cache/reflector.go:105: Failed to watch *v1.Node: Get http://127.0.0.1:42849/api/v1/nodes?allowWatchBookmarks=true&resourceVersion=1217&timeout=10s&timeoutSeconds=540&watch=true: dial tcp 127.0.0.1:42849: connect: connection refused
ok  	sigs.k8s.io/cluster-api/controllers	80.710s
=== RUN   TestGetResourceFound
--- PASS: TestGetResourceFound (0.00s)
=== RUN   TestGetResourceNotFound
--- PASS: TestGetResourceNotFound (0.00s)
=== RUN   TestCloneTemplateResourceNotFound
... skipping 256 lines ...
I0728 16:02:55.773921   11046 webhook.go:123] controller-runtime/builder "msg"="Registering a validating webhook"  "GVK"={"Group":"addons.cluster.x-k8s.io","Version":"v1alpha3","Kind":"ClusterResourceSet"} "path"="/validate-addons-cluster-x-k8s-io-v1alpha3-clusterresourceset"
I0728 16:02:55.773943   11046 server.go:121] controller-runtime/webhook "msg"="registering webhook"  "path"="/validate-addons-cluster-x-k8s-io-v1alpha3-clusterresourceset"
I0728 16:02:55.774247   11046 server.go:141] controller-runtime/webhook/webhooks "msg"="starting webhook server"  
I0728 16:02:55.775172   11046 certwatcher.go:127] controller-runtime/certwatcher "msg"="Updated current TLS certificate"  
I0728 16:02:55.775393   11046 server.go:199] controller-runtime/webhook "msg"="serving webhook server"  "host"="" "port"=36533
I0728 16:02:55.775683   11046 certwatcher.go:83] controller-runtime/certwatcher "msg"="Starting certificate watcher"  
E0728 16:02:57.953565   11046 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="cluster-cache-test-gh86c/test-cluster"
•E0728 16:02:58.451713   11046 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="the server could not find the requested resource"  "cluster"="cluster-cache-test-ddp5n/test-cluster"
•E0728 16:02:59.030521   11046 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="Get http://127.0.0.1:33337/?timeout=50ms: dial tcp 127.0.0.1:33337: connect: connection refused"  "cluster"="cluster-cache-test-8qkmq/test-cluster"
•I0728 16:02:59.267179   11046 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="cluster" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"controlPlaneEndpoint":{"host":"","port":0}},"status":{"infrastructureReady":false,"controlPlaneInitialized":false}}}
I0728 16:02:59.367653   11046 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="cluster" 
I0728 16:02:59.367713   11046 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="cluster" "worker count"=1
•I0728 16:02:59.539913   11046 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="cluster" 
I0728 16:02:59.553426   11046 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinedeployment" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"clusterName":"","selector":{},"template":{"metadata":{},"spec":{"clusterName":"","bootstrap":{},"infrastructureRef":{}}}},"status":{}}}
I0728 16:02:59.753808   11046 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="machinedeployment" 
I0728 16:02:59.753874   11046 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="machinedeployment" "worker count"=1
I0728 16:02:59.753938   11046 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinedeployment" "source"={}
•E0728 16:03:00.278180   11046 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0728 16:03:00.278247   11046 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-587866800/tls.crt: no such file or directory"  
I0728 16:03:00.278297   11046 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  
I0728 16:03:00.278477   11046 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machinedeployment" 
E0728 16:03:00.325538   11046 reflector.go:309] pkg/mod/k8s.io/client-go@v0.17.9/tools/cache/reflector.go:105: Failed to watch *v1alpha3.Cluster: Get http://127.0.0.1:35851/apis/cluster.x-k8s.io/v1alpha3/clusters?allowWatchBookmarks=true&resourceVersion=146&timeout=10s&timeoutSeconds=478&watch=true: dial tcp 127.0.0.1:35851: connect: connection refused


Ran 5 of 5 Specs in 15.681 seconds
SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestGinkgoSuite (15.68s)
PASS
ok  	sigs.k8s.io/cluster-api/controllers/remote	15.858s
?   	sigs.k8s.io/cluster-api/controllers/remote/fake	[no test files]
=== RUN   TestKubeadmControlPlaneDefault
=== RUN   TestKubeadmControlPlaneDefault/for_KubeadmControlPLane
... skipping 154 lines ...
=== RUN   TestPaths/empty_input_makes_for_empty_output
--- PASS: TestPaths (0.00s)
    --- PASS: TestPaths/basic_check (0.00s)
    --- PASS: TestPaths/empty_input_makes_for_empty_output (0.00s)
PASS
ok  	sigs.k8s.io/cluster-api/controlplane/kubeadm/api/v1alpha3	0.124s
FAIL	sigs.k8s.io/cluster-api/controlplane/kubeadm/controllers [build failed]
FAIL	sigs.k8s.io/cluster-api/controlplane/kubeadm/internal [build failed]
=== RUN   TestEtcdMembers_WithErrors
--- PASS: TestEtcdMembers_WithErrors (0.00s)
=== RUN   TestEtcdMembers_WithSuccess
--- PASS: TestEtcdMembers_WithSuccess (0.00s)
PASS
ok  	sigs.k8s.io/cluster-api/controlplane/kubeadm/internal/etcd	0.037s
... skipping 280 lines ...
I0728 16:03:09.908283   11806 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="clusterresourceset" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"controlPlaneEndpoint":{"host":"","port":0}},"status":{"infrastructureReady":false,"controlPlaneInitialized":false}}}
I0728 16:03:09.908403   11806 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="clusterresourceset" "source"={"Type":{"kind":"ConfigMap","apiVersion":"v1","metadata":{"creationTimestamp":null}}}
I0728 16:03:10.009233   11806 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="clusterresourceset" "source"={"Type":{"kind":"Secret","apiVersion":"v1","metadata":{"creationTimestamp":null}}}
I0728 16:03:10.109609   11806 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="clusterresourceset" 
I0728 16:03:10.109682   11806 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="clusterresourceset" "worker count"=1
I0728 16:03:10.198210   11806 clusterresourcesetbinding_controller.go:84]  "msg"="deleting ClusterResourceSetBinding because the owner Cluster no longer exists" "clusterresourcesetbinding"={"Namespace":"default","Name":"cluster-newrcf"} 
E0728 16:03:10.224560   11806 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•I0728 16:03:11.275445   11806 clusterresourcesetbinding_controller.go:84]  "msg"="deleting ClusterResourceSetBinding because the owner Cluster no longer exists" "clusterresourcesetbinding"={"Namespace":"default","Name":"cluster-ms2ma4"} 
E0728 16:03:11.310006   11806 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•E0728 16:03:12.372187   11806 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get data field from the resource" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
E0728 16:03:13.394737   11806 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•E0728 16:03:14.550554   11806 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•E0728 16:03:15.589513   11806 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset2\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset2" "namespace"="default"
•I0728 16:03:16.613055   11806 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  
I0728 16:03:16.613170   11806 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="clusterresourceset" 
I0728 16:03:16.613283   11806 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="clusterresourcesetbinding" 
E0728 16:03:16.620180   11806 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="Patch http://127.0.0.1:41025/apis/addons.cluster.x-k8s.io/v1alpha3/namespaces/default/clusterresourcesets/test-clusterresourceset/status: dial tcp 127.0.0.1:41025: connect: connection refused" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"


Ran 5 of 5 Specs in 20.074 seconds
SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestAPIs (20.07s)
PASS
ok  	sigs.k8s.io/cluster-api/exp/addons/controllers	20.167s
=== RUN   TestMachinePoolDefault
=== RUN   TestMachinePoolDefault/for_MachinePool
=== RUN   TestMachinePoolDefault/for_MachinePool/validate-on-create
... skipping 60 lines ...
    --- PASS: TestReconcileMachinePoolBootstrap/new_machinepool,_no_bootstrap_config_or_data (0.00s)
    --- PASS: TestReconcileMachinePoolBootstrap/existing_machinepool,_bootstrap_data_should_not_change (0.00s)
    --- PASS: TestReconcileMachinePoolBootstrap/existing_machinepool,_bootstrap_provider_is_to_not_ready (0.00s)
=== RUN   TestReconcileMachinePoolInfrastructure
=== RUN   TestReconcileMachinePoolInfrastructure/new_machinepool,_infrastructure_config_ready
=== RUN   TestReconcileMachinePoolInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machinepool_is_running,_infra_object_is_deleted,_expect_failed
E0728 16:02:58.134530   11906 machinepool_controller_phases.go:237]  "msg"="MachinePool infrastructure reference has been deleted after being ready, setting failure state" "error"="could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureConfig \"infra-config1\" for MachinePool \"machinepool-test\" in namespace \"default\", requeuing: failed to retrieve InfrastructureConfig external object \"default\"/\"infra-config1\": infrastructureconfigs.infrastructure.cluster.x-k8s.io \"infra-config1\" not found" "cluster"="test-cluster" "machinepool"="machinepool-test" "namespace"="default" 
=== RUN   TestReconcileMachinePoolInfrastructure/infrastructure_ref_is_paused
--- PASS: TestReconcileMachinePoolInfrastructure (0.00s)
    --- PASS: TestReconcileMachinePoolInfrastructure/new_machinepool,_infrastructure_config_ready (0.00s)
    --- PASS: TestReconcileMachinePoolInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machinepool_is_running,_infra_object_is_deleted,_expect_failed (0.00s)
    --- PASS: TestReconcileMachinePoolInfrastructure/infrastructure_ref_is_paused (0.00s)
=== RUN   TestMachinePoolFinalizer
... skipping 92 lines ...
•I0728 16:03:13.790499   11906 machinepool_controller_noderef.go:90]  "msg"="Cannot assign NodeRefs to MachinePool, no matching Nodes"  
•I0728 16:03:13.807791   11906 machinepool_controller_noderef.go:90]  "msg"="Cannot assign NodeRefs to MachinePool, no matching Nodes"  
•I0728 16:03:13.816669   11906 machinepool_controller_phases.go:268]  "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "machinepool"="machinepool-test" "namespace"="default" 
•I0728 16:03:13.843020   11906 machinepool_controller_noderef.go:90]  "msg"="Cannot assign NodeRefs to MachinePool, no matching Nodes"  
•I0728 16:03:13.854389   11906 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinepool" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"controlPlaneEndpoint":{"host":"","port":0}},"status":{"infrastructureReady":false,"controlPlaneInitialized":false}}}
I0728 16:03:13.876140   11906 machinepool_controller_noderef.go:90]  "msg"="Cannot assign NodeRefs to MachinePool, no matching Nodes"  
••E0728 16:03:13.879162   11906 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0728 16:03:13.879201   11906 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-987199522/tls.crt: no such file or directory"  
E0728 16:03:13.879220   11906 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0728 16:03:13.879237   11906 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-987199522/tls.crt: no such file or directory"  
I0728 16:03:13.879658   11906 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  


Ran 9 of 9 Specs in 15.765 seconds
SUCCESS! -- 9 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestAPIs (15.77s)
PASS
ok  	sigs.k8s.io/cluster-api/exp/controllers	16.022s
=== RUN   TestMustDefaultConfig
--- PASS: TestMustDefaultConfig (0.00s)
=== RUN   TestObjectToKind
... skipping 244 lines ...
--- PASS: TestGetFirstReasonAndMessage (0.00s)
=== RUN   TestNewConditionsGroup
--- PASS: TestNewConditionsGroup (0.00s)
=== RUN   TestMergeRespectPriority
=== RUN   TestMergeRespectPriority/aggregate_nil_list_return_nil
=== RUN   TestMergeRespectPriority/aggregate_empty_list_return_nil
=== RUN   TestMergeRespectPriority/When_there_is_false/error_it_returns_false/error
=== RUN   TestMergeRespectPriority/When_there_is_false/warning_and_no_false/error,_it_returns_false/warning
=== RUN   TestMergeRespectPriority/When_there_is_false/info_and_no_false/error_or_false/warning,_it_returns_false/info
=== RUN   TestMergeRespectPriority/When_there_is_true_and_no_false/*,_it_returns_info
=== RUN   TestMergeRespectPriority/When_there_is_unknown_and_no_true_or_false/*,_it_returns_unknown
=== RUN   TestMergeRespectPriority/nil_conditions_are_ignored
--- PASS: TestMergeRespectPriority (0.00s)
    --- PASS: TestMergeRespectPriority/aggregate_nil_list_return_nil (0.00s)
    --- PASS: TestMergeRespectPriority/aggregate_empty_list_return_nil (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_false/error_it_returns_false/error (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_false/warning_and_no_false/error,_it_returns_false/warning (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_false/info_and_no_false/error_or_false/warning,_it_returns_false/info (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_true_and_no_false/*,_it_returns_info (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_unknown_and_no_true_or_false/*,_it_returns_unknown (0.00s)
    --- PASS: TestMergeRespectPriority/nil_conditions_are_ignored (0.00s)
=== RUN   TestNewPatch
=== RUN   TestNewPatch/No_changes_return_empty_patch
... skipping 203 lines ...
I0728 16:03:34.583360   14824 server.go:199] controller-runtime/webhook "msg"="serving webhook server"  "host"="" "port"=44813
I0728 16:03:34.583492   14824 certwatcher.go:83] controller-runtime/certwatcher "msg"="Starting certificate watcher"  
••••••••••••••I0728 16:03:35.515197   14824 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  


Ran 14 of 14 Specs in 7.638 seconds
SUCCESS! -- 14 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestPatch (7.64s)
=== RUN   TestToUnstructured
=== RUN   TestToUnstructured/with_a_typed_object
=== RUN   TestToUnstructured/with_an_unstructured_object
--- PASS: TestToUnstructured (0.00s)
    --- PASS: TestToUnstructured/with_a_typed_object (0.00s)
... skipping 103 lines ...
    --- PASS: TestToUnstructured/returns_error_for_invalid_yaml#01 (0.00s)
    --- PASS: TestToUnstructured/returns_error_for_invalid_yaml#02 (0.00s)
=== RUN   TestFromUnstructured
--- PASS: TestFromUnstructured (0.00s)
PASS
ok  	sigs.k8s.io/cluster-api/util/yaml	0.038s
make: *** [Makefile:116: test] Error 2
+ EXIT_VALUE=2
+ set +o xtrace