This job view page is being replaced by Spyglass soon. Check out the new job view.
PRsbueringer: ✨ support NodeRegistrationOptions.IgnorePreflightErrors
ResultABORTED
Tests 32 failed / 1611 succeeded
Started2021-07-29 08:24
Elapsed3m17s
Revisiond41926b39072762445bf0891f04ca6e673722171
Refs 4905

Test Failures


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion$'
=== RUN   TestFuzzyConversion
=== RUN   TestFuzzyConversion/for_Cluster
=== RUN   TestFuzzyConversion/for_Cluster/spoke-hub-spoke
2021/07/29 08:26:03 http: TLS handshake error from 127.0.0.1:33046: EOF
=== RUN   TestFuzzyConversion/for_Cluster/hub-spoke-hub
=== RUN   TestFuzzyConversion/for_Machine
=== RUN   TestFuzzyConversion/for_Machine/spoke-hub-spoke
=== RUN   TestFuzzyConversion/for_Machine/hub-spoke-hub
=== RUN   TestFuzzyConversion/for_MachineSet
=== RUN   TestFuzzyConversion/for_MachineSet/spoke-hub-spoke
=== RUN   TestFuzzyConversion/for_MachineSet/hub-spoke-hub
=== RUN   TestFuzzyConversion/for_MachineDeployment
=== RUN   TestFuzzyConversion/for_MachineDeployment/spoke-hub-spoke
=== RUN   TestFuzzyConversion/for_MachineDeployment/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_Cluster 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_Cluster$'
=== RUN   TestFuzzyConversion/for_Cluster

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_Cluster/hub-spoke-hub 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_Cluster\/hub\-spoke\-hub$'
=== RUN   TestFuzzyConversion/for_Cluster/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_Cluster/spoke-hub-spoke 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_Cluster\/spoke\-hub\-spoke$'
=== RUN   TestFuzzyConversion/for_Cluster/spoke-hub-spoke
2021/07/29 08:26:03 http: TLS handshake error from 127.0.0.1:33046: EOF

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_Machine 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_Machine$'
=== RUN   TestFuzzyConversion/for_Machine

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_Machine/hub-spoke-hub 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_Machine\/hub\-spoke\-hub$'
=== RUN   TestFuzzyConversion/for_Machine/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_Machine/spoke-hub-spoke 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_Machine\/spoke\-hub\-spoke$'
=== RUN   TestFuzzyConversion/for_Machine/spoke-hub-spoke

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_MachineDeployment 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_MachineDeployment$'
=== RUN   TestFuzzyConversion/for_MachineDeployment

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_MachineDeployment/hub-spoke-hub 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_MachineDeployment\/hub\-spoke\-hub$'
=== RUN   TestFuzzyConversion/for_MachineDeployment/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_MachineDeployment/spoke-hub-spoke 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_MachineDeployment\/spoke\-hub\-spoke$'
=== RUN   TestFuzzyConversion/for_MachineDeployment/spoke-hub-spoke

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_MachineSet 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_MachineSet$'
=== RUN   TestFuzzyConversion/for_MachineSet

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_MachineSet/hub-spoke-hub 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_MachineSet\/hub\-spoke\-hub$'
=== RUN   TestFuzzyConversion/for_MachineSet/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/api/v1alpha3 TestFuzzyConversion/for_MachineSet/spoke-hub-spoke 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/api\/v1alpha3\sTestFuzzyConversion\/for\_MachineSet\/spoke\-hub\-spoke$'
=== RUN   TestFuzzyConversion/for_MachineSet/spoke-hub-spoke

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1alpha3 TestFuzzyConversion 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/bootstrap\/kubeadm\/api\/v1alpha3\sTestFuzzyConversion$'
=== RUN   TestFuzzyConversion
=== RUN   TestFuzzyConversion/for_KubeadmConfig
=== RUN   TestFuzzyConversion/for_KubeadmConfig/spoke-hub-spoke
=== RUN   TestFuzzyConversion/for_KubeadmConfig/hub-spoke-hub
=== RUN   TestFuzzyConversion/for_KubeadmConfigTemplate
=== RUN   TestFuzzyConversion/for_KubeadmConfigTemplate/spoke-hub-spoke
=== RUN   TestFuzzyConversion/for_KubeadmConfigTemplate/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1alpha3 TestFuzzyConversion/for_KubeadmConfig 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/bootstrap\/kubeadm\/api\/v1alpha3\sTestFuzzyConversion\/for\_KubeadmConfig$'
=== RUN   TestFuzzyConversion/for_KubeadmConfig

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1alpha3 TestFuzzyConversion/for_KubeadmConfig/hub-spoke-hub 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/bootstrap\/kubeadm\/api\/v1alpha3\sTestFuzzyConversion\/for\_KubeadmConfig\/hub\-spoke\-hub$'
=== RUN   TestFuzzyConversion/for_KubeadmConfig/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1alpha3 TestFuzzyConversion/for_KubeadmConfig/spoke-hub-spoke 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/bootstrap\/kubeadm\/api\/v1alpha3\sTestFuzzyConversion\/for\_KubeadmConfig\/spoke\-hub\-spoke$'
=== RUN   TestFuzzyConversion/for_KubeadmConfig/spoke-hub-spoke

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1alpha3 TestFuzzyConversion/for_KubeadmConfigTemplate 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/bootstrap\/kubeadm\/api\/v1alpha3\sTestFuzzyConversion\/for\_KubeadmConfigTemplate$'
=== RUN   TestFuzzyConversion/for_KubeadmConfigTemplate

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1alpha3 TestFuzzyConversion/for_KubeadmConfigTemplate/hub-spoke-hub 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/bootstrap\/kubeadm\/api\/v1alpha3\sTestFuzzyConversion\/for\_KubeadmConfigTemplate\/hub\-spoke\-hub$'
=== RUN   TestFuzzyConversion/for_KubeadmConfigTemplate/hub-spoke-hub

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/bootstrap/kubeadm/api/v1alpha3 TestFuzzyConversion/for_KubeadmConfigTemplate/spoke-hub-spoke 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/bootstrap\/kubeadm\/api\/v1alpha3\sTestFuzzyConversion\/for\_KubeadmConfigTemplate\/spoke\-hub\-spoke$'
=== RUN   TestFuzzyConversion/for_KubeadmConfigTemplate/spoke-hub-spoke

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile$'
=== RUN   TestMachineHealthCheck_Reconcile
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_no_existing_labels_exist
E0729 08:26:57.976502   13091 controller.go:304] controller-runtime/manager/controller/machinedeployment "msg"="Reconciler error" "error"="machinedeployments.cluster.x-k8s.io \"md-57h7g\" not found" "name"="md-57h7g" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineDeployment" 
I0729 08:26:57.999117   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:57.999446   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.021017   13091 machineset_controller.go:415] controller-runtime/manager/controller/machineset "msg"="Deleted machine" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.027599   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359255ef5cf3", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"SuccessfulDelete", Message:"Deleted machine \"md-57h7g-5575b6c97-hkppq\"", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774797972f3, ext:30666244908, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774814159f6, ext:30723044432, loc:(*time.Location)(0x2bdcb20)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359255ef5cf3" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.043129   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_the_label_has_the_wrong_value
E0729 08:26:58.112529   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-rn42k\" not found" "name"="test-cluster-rn42k" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:58.171961   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.173009   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-rn42k\" not found"  "cluster"="test-mhc-jsszl/test-cluster-rn42k"
E0729 08:26:58.190482   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-rrjb2\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-rrjb2\" not found]" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.190551   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.195849   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.226851   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.268899   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-5kkmb" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.277367   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-vc7jp: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test2-vc7jp" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
E0729 08:26:58.297661   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.297982   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster" "name"="md-57h7g-5575b6c97-hkppq" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"md-57h7g-5575b6c97-hkppq-9qrsc","uid":"6fadd2e2-060b-4360-b8da-7e13f2fc76bf","apiVersion":"v1"}
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_other_labels_are_present
E0729 08:26:58.333059   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-8bqsk\" not found" "name"="test-cluster-8bqsk" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
E0729 08:26:58.370768   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-9bkkk: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test4-9bkkk" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
E0729 08:26:58.402899   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-8bqsk\" not found"  "cluster"="test-mhc-vwrcx/test-cluster-8bqsk"
I0729 08:26:58.423344   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:58.423403   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.429786   13091 machineset_controller.go:415] controller-runtime/manager/controller/machineset "msg"="Deleted machine" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.435079   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359255ef5cf3", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"SuccessfulDelete", Message:"Deleted machine \"md-57h7g-5575b6c97-hkppq\"", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774797972f3, ext:30666244908, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774999ec101, ext:31131818820, loc:(*time.Location)(0x2bdcb20)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359255ef5cf3" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.529542   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.531069   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-8bqsk\" not found"  "cluster"="test-mhc-vwrcx/test-cluster-8bqsk"
I0729 08:26:58.549133   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:58.549209   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.558459   13091 machineset_controller.go:415] controller-runtime/manager/controller/machineset "msg"="Deleted machine" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.561883   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-5kkmb\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-5kkmb\" not found]" "name"="test-mhc-5kkmb" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.561971   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.573022   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359255ef5cf3", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"SuccessfulDelete", Message:"Deleted machine \"md-57h7g-5575b6c97-hkppq\"", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774797972f3, ext:30666244908, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774a14a25b4, ext:31260491756, loc:(*time.Location)(0x2bdcb20)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359255ef5cf3" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_no_existing_ones_exist
E0729 08:26:58.642798   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-jk6g6\" not found" "name"="test-cluster-jk6g6" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:58.710849   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.712190   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-jk6g6\" not found"  "cluster"="test-mhc-274lr/test-cluster-jk6g6"
E0729 08:26:58.720930   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-4z9jq\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-4z9jq\" not found]" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.721151   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-5kkmb" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.721535   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.723105   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.723629   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster" "name"="md-57h7g-5575b6c97-hkppq" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"md-57h7g-5575b6c97-hkppq-9qrsc","uid":"6fadd2e2-060b-4360-b8da-7e13f2fc76bf","apiVersion":"v1"}
I0729 08:26:58.726264   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.781200   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:58.781414   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.799884   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.802435   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97-hkppq" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:26:58.802808   13091 machineset_controller.go:410] controller-runtime/manager/controller/machineset "msg"="Unable to delete Machine" "error"="machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.806571   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359287edcff7", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"FailedDelete", Message:"Failed to delete machine \"md-57h7g-5575b6c97-hkppq\": machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774afdd1bf7, ext:31505004080, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774afdd1bf7, ext:31505004080, loc:(*time.Location)(0x2bdcb20)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359287edcff7" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.827096   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.827157   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.852180   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test6-lrz6f" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=1
I0729 08:26:58.852309   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test6-lrz6f" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test6-v6pw9" "indirect descendants count"=0
I0729 08:26:58.932155   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.933072   13091 machineset_controller.go:159] controller-runtime/manager/controller/machineset "msg"="Failed to reconcile MachineSet" "error"="failed to sync MachineSet replicas: machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" 
E0729 08:26:58.933508   13091 controller.go:304] controller-runtime/manager/controller/machineset "msg"="Reconciler error" "error"="failed to sync MachineSet replicas: machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" 
E0729 08:26:58.936721   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.169635928fae9594", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"ReconcileError", Message:"failed to sync MachineSet replicas: machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774b79de194, ext:31635078088, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774b79de194, ext:31635078088, loc:(*time.Location)(0x2bdcb20)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.169635928fae9594" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.961619   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.961692   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.974656   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.006285   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_modifying_existing_ones
I0729 08:26:59.041965   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:59.057713   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:26:59.109690   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-tltwd\" not found" "name"="test-cluster-tltwd" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:59.152110   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.286417   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
I0729 08:26:59.318817   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.332634   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.372995   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
=== RUN   TestMachineHealthCheck_Reconcile/it_ignores_Machines_not_matching_the_label_selector
E0729 08:26:59.425163   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-hp4hk\" not found" "name"="test-cluster-hp4hk" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:59.486436   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-vn275
I0729 08:26:59.656601   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:59.855084   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.867868   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.867940   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.868823   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.869831   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
machine created: test-mhc-machine-vrrqw
I0729 08:26:59.903855   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.914394   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.929909   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.944222   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.963007   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.976993   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.977277   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
node created: test-mhc-node-rj8zn
I0729 08:26:59.985683   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-q2pg2" "machine"="test-mhc-machine-vrrqw" "name"="test-mhc-machine-vrrqw" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-rj8zn"
I0729 08:27:00.009679   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-x4xc9
machine created: test-mhc-machine-z7v7n
I0729 08:27:00.081121   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.081550   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.131315   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.131971   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.248544   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.249132   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.262505   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.263080   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.276167   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.276845   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.293166   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.293725   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.307942   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.308616   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.312450   13091 controller.go:130] controller-runtime/manager/controller/machine "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "source"={}
I0729 08:27:00.380885   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.381816   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.406408   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.407081   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
node created: test-mhc-node-kkr8b
I0729 08:27:00.435470   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.436354   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.442561   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:27:00.455748   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
inframachine created: test-mhc-machine-infra-4vqrx
I0729 08:27:00.495808   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
machine created: test-mhc-machine-c5pwz
E0729 08:27:00.496183   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:00.524493   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test6-lrz6f" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"id-node-1","uid":"39025841-0a60-407e-9ca9-925d79716290","apiVersion":"v1"}
E0729 08:27:00.525847   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="failed to get /, Kind= \"\" for Machine \"test6-v6pw9\" in namespace \"default\": failed to retrieve  external object \"default\"/\"\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:00.604324   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.640400   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.662473   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
node created: test-mhc-node-nrbfg
I0729 08:27:00.811985   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-q2pg2" "machine"="test-mhc-machine-c5pwz" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-nrbfg"
inframachine created: test-mhc-machine-infra-lwx2t
machine created: test-mhc-machine-stq22
I0729 08:27:00.909350   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:00.909426   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
node created: test-mhc-node-hszf7
Cleaning up nodes, machines and infra machines.
I0729 08:27:01.187187   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.187251   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:01.204743   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.205887   13091 machine_controller.go:302] controller-runtime/manager/controller/machine "msg"="Draining node" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "node"="test-mhc-node-nrbfg"
Cleaning up nodes, machines and infra machines.
I0729 08:27:01.246778   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.247408   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-vrrqw/"
E0729 08:27:01.252087   13091 machine_controller.go:488] controller-runtime/manager/controller/machine "msg"="Could not find node from noderef, it may have already been deleted" "error"="nodes \"test-mhc-node-nrbfg\" not found" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "node"="test-mhc-node-nrbfg" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.288940   13091 machine_controller.go:359] controller-runtime/manager/controller/machine "msg"="Deleting node" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "node"="test-mhc-node-nrbfg"
I0729 08:27:01.303053   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.303762   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-vrrqw/"
I0729 08:27:01.304384   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:01.338320   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=4
I0729 08:27:01.338390   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-vrrqw,test-mhc-machine-z7v7n;Worker machines: test-mhc-machine-stq22,test-mhc-machine-c5pwz" "indirect descendants count"=0
I0729 08:27:01.350248   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.350940   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-vrrqw/"
I0729 08:27:01.351514   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
E0729 08:27:01.352872   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-c5pwz\" not found" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.353096   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-stq22" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-hszf7"}
I0729 08:27:01.358913   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
=== RUN   TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_cluster_infrastructure_is_not_ready
I0729 08:27:01.361877   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=3
I0729 08:27:01.361950   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-vrrqw,test-mhc-machine-z7v7n;Worker machines: test-mhc-machine-stq22" "indirect descendants count"=0
I0729 08:27:01.463329   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-w8vrr
E0729 08:27:01.507640   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-stq22\" not found" "name"="test-mhc-machine-stq22" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.507976   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-vrrqw" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"test-mhc-node-rj8zn","uid":"9351cbc6-9dec-4eb1-8936-9f9fcf48b011","apiVersion":"v1"}
machine created: test-mhc-machine-wqwzx
I0729 08:27:01.588648   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:27:01.595629   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-vrrqw\" not found" "name"="test-mhc-machine-vrrqw" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.595924   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-z7v7n" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-kkr8b"}
I0729 08:27:01.646603   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:27:01.678908   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-z7v7n\" not found" "name"="test-mhc-machine-z7v7n" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.695826   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.704626   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.733933   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
node created: test-mhc-node-h7wc7
I0729 08:27:01.757672   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.766241   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-g2zjk
I0729 08:27:01.782443   13091 controller.go:130] controller-runtime/manager/controller/machine "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "source"={}
machine created: test-mhc-machine-rmf2z
I0729 08:27:01.835383   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test6-lrz6f" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"id-node-1","uid":"39025841-0a60-407e-9ca9-925d79716290","apiVersion":"v1"}
E0729 08:27:01.836569   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="failed to get /, Kind= \"\" for Machine \"test6-v6pw9\" in namespace \"default\": failed to retrieve  external object \"default\"/\"\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:01.884384   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:01.901098   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.943746   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.954391   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.969691   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.969997   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
node created: test-mhc-node-hqvtc
I0729 08:27:02.022963   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.046407   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.053046   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.063850   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.075978   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-tb4cc" "machine"="test-mhc-machine-rmf2z" "name"="test-mhc-machine-rmf2z" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-hqvtc"
I0729 08:27:02.099386   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.111360   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
Cleaning up nodes, machines and infra machines.
I0729 08:27:02.172927   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.175525   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-tb4cc" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-gpxtw/test-mhc-9pb9t/test-mhc-machine-wqwzx/"
E0729 08:27:02.241464   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"test-mhc-machine-wqwzx\" in namespace \"test-mhc-gpxtw\": cannot find node with matching ProviderID" "name"="test-mhc-machine-wqwzx" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.241873   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-tb4cc" "name"="test-mhc-machine-rmf2z" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="no control plane members" "node"={"kind":"Node","name":"test-mhc-node-hqvtc","uid":"a591e025-dd49-462a-817a-3738f82e577b","apiVersion":"v1"}
I0729 08:27:02.247841   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=2
I0729 08:27:02.247910   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-rmf2z,test-mhc-machine-wqwzx" "indirect descendants count"=0
I0729 08:27:02.261896   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=2
I0729 08:27:02.261981   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-rmf2z,test-mhc-machine-wqwzx" "indirect descendants count"=0
=== RUN   TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_all_Machines_are_healthy
E0729 08:27:02.335808   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-rmf2z\" not found" "name"="test-mhc-machine-rmf2z" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.344686   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.344767   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.345703   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-tb4cc" "name"="test-mhc-machine-wqwzx" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-h7wc7"}
inframachine created: test-mhc-machine-infra-hww2c
machine created: test-mhc-machine-4lgs6
E0729 08:27:02.408810   13091 machinehealthcheck_controller.go:166] controller-runtime/manager/controller/machinehealthcheck "msg"="Failed to reconcile MachineHealthCheck" "error"="failed to patch unhealthy machine status for machine: test-mhc-gpxtw/test-mhc-machine-wqwzx: machines.cluster.x-k8s.io \"test-mhc-machine-wqwzx\" not found" "cluster"="test-cluster-tb4cc" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:27:02.413376   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-wqwzx\" not found" "name"="test-mhc-machine-wqwzx" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:02.414942   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[failed to patch unhealthy machine status for machine: test-mhc-gpxtw/test-mhc-machine-wqwzx: machines.cluster.x-k8s.io \"test-mhc-machine-wqwzx\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-9pb9t\" not found]" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.415014   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
node created: test-mhc-node-9f4h6
inframachine created: test-mhc-machine-infra-j82px
machine created: test-mhc-machine-w2zfn
I0729 08:27:02.550450   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
I0729 08:27:02.551721   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-z7wzf" "machine"="test-mhc-machine-4lgs6" "name"="test-mhc-machine-4lgs6" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-9f4h6"
I0729 08:27:02.619285   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.619385   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.619943   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.634121   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.634749   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.636722   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.637749   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.646005   13091 controller.go:130] controller-runtime/manager/controller/machine "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "source"={}
I0729 08:27:02.668793   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.669434   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.688304   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.689062   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.719678   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.720317   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
node created: test-mhc-node-m4gx5
I0729 08:27:02.740054   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.741299   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.756140   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.756859   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.770920   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.774192   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test6-lrz6f" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"id-node-1","uid":"39025841-0a60-407e-9ca9-925d79716290","apiVersion":"v1"}
E0729 08:27:02.775582   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="failed to get /, Kind= \"\" for Machine \"test6-v6pw9\" in namespace \"default\": failed to retrieve  external object \"default\"/\"\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:02.784553   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:02.796488   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.818099   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.832854   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.842376   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.844691   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.851071   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.867302   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
Cleaning up nodes, machines and infra machines.
I0729 08:27:02.878443   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.884350   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.884966   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-4lgs6/"
I0729 08:27:02.931569   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.932242   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-4lgs6/"
I0729 08:27:02.932828   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.966675   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Machine sets: md-57h7g-684b9cfc89,md-57h7g-5575b6c97;Worker machines: md-57h7g-65496964d6-9xkgf,md-57h7g-684b9cfc89-9znlc,md-57h7g-684b9cfc89-kf4h4,md-57h7g-684b9cfc89-94pvx,md-57h7g-65496964d6-bh8pt" "indirect descendants count"=7
E0729 08:27:02.974011   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"test-mhc-machine-4lgs6\" in namespace \"test-mhc-4g22p\": cannot find node with matching ProviderID" "name"="test-mhc-machine-4lgs6" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.974413   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-z7wzf" "name"="test-mhc-machine-w2zfn" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="no control plane members" "node"={"name":"test-mhc-node-m4gx5"}
I0729 08:27:02.981184   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-z7wzf" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=2
I0729 08:27:02.981498   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-z7wzf" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-w2zfn,test-mhc-machine-4lgs6" "indirect descendants count"=0

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_all_Machines_are_healthy 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_doesn\'t\_mark\_anything\_unhealthy\_when\_all\_Machines\_are\_healthy$'
=== RUN   TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_all_Machines_are_healthy
E0729 08:27:02.335808   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-rmf2z\" not found" "name"="test-mhc-machine-rmf2z" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.344686   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.344767   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.345703   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-tb4cc" "name"="test-mhc-machine-wqwzx" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-h7wc7"}
inframachine created: test-mhc-machine-infra-hww2c
machine created: test-mhc-machine-4lgs6
E0729 08:27:02.408810   13091 machinehealthcheck_controller.go:166] controller-runtime/manager/controller/machinehealthcheck "msg"="Failed to reconcile MachineHealthCheck" "error"="failed to patch unhealthy machine status for machine: test-mhc-gpxtw/test-mhc-machine-wqwzx: machines.cluster.x-k8s.io \"test-mhc-machine-wqwzx\" not found" "cluster"="test-cluster-tb4cc" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:27:02.413376   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-wqwzx\" not found" "name"="test-mhc-machine-wqwzx" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:02.414942   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[failed to patch unhealthy machine status for machine: test-mhc-gpxtw/test-mhc-machine-wqwzx: machines.cluster.x-k8s.io \"test-mhc-machine-wqwzx\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-9pb9t\" not found]" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.415014   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
node created: test-mhc-node-9f4h6
inframachine created: test-mhc-machine-infra-j82px
machine created: test-mhc-machine-w2zfn
I0729 08:27:02.550450   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
I0729 08:27:02.551721   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-z7wzf" "machine"="test-mhc-machine-4lgs6" "name"="test-mhc-machine-4lgs6" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-9f4h6"
I0729 08:27:02.619285   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.619385   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.619943   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.634121   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.634749   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.636722   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.637749   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.646005   13091 controller.go:130] controller-runtime/manager/controller/machine "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "source"={}
I0729 08:27:02.668793   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.669434   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.688304   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.689062   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.719678   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.720317   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
node created: test-mhc-node-m4gx5
I0729 08:27:02.740054   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.741299   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.756140   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.756859   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.770920   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.774192   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test6-lrz6f" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"id-node-1","uid":"39025841-0a60-407e-9ca9-925d79716290","apiVersion":"v1"}
E0729 08:27:02.775582   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="failed to get /, Kind= \"\" for Machine \"test6-v6pw9\" in namespace \"default\": failed to retrieve  external object \"default\"/\"\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:02.784553   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:02.796488   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.818099   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.832854   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.842376   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.844691   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.851071   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.867302   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
Cleaning up nodes, machines and infra machines.
I0729 08:27:02.878443   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.884350   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.884966   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-4lgs6/"
I0729 08:27:02.931569   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.932242   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-4lgs6/"
I0729 08:27:02.932828   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-z7wzf" "name"="test-mhc-pfwzj" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-4g22p/test-mhc-pfwzj/test-mhc-machine-w2zfn/"
I0729 08:27:02.966675   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Machine sets: md-57h7g-684b9cfc89,md-57h7g-5575b6c97;Worker machines: md-57h7g-65496964d6-9xkgf,md-57h7g-684b9cfc89-9znlc,md-57h7g-684b9cfc89-kf4h4,md-57h7g-684b9cfc89-94pvx,md-57h7g-65496964d6-bh8pt" "indirect descendants count"=7
E0729 08:27:02.974011   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"test-mhc-machine-4lgs6\" in namespace \"test-mhc-4g22p\": cannot find node with matching ProviderID" "name"="test-mhc-machine-4lgs6" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.974413   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-z7wzf" "name"="test-mhc-machine-w2zfn" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="no control plane members" "node"={"name":"test-mhc-node-m4gx5"}
I0729 08:27:02.981184   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-z7wzf" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=2
I0729 08:27:02.981498   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-z7wzf" "namespace"="test-mhc-4g22p" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-w2zfn,test-mhc-machine-4lgs6" "indirect descendants count"=0

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_cluster_infrastructure_is_not_ready 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_doesn\'t\_mark\_anything\_unhealthy\_when\_cluster\_infrastructure\_is\_not\_ready$'
=== RUN   TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_cluster_infrastructure_is_not_ready
I0729 08:27:01.361877   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=3
I0729 08:27:01.361950   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-vrrqw,test-mhc-machine-z7v7n;Worker machines: test-mhc-machine-stq22" "indirect descendants count"=0
I0729 08:27:01.463329   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-w8vrr
E0729 08:27:01.507640   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-stq22\" not found" "name"="test-mhc-machine-stq22" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.507976   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-vrrqw" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"test-mhc-node-rj8zn","uid":"9351cbc6-9dec-4eb1-8936-9f9fcf48b011","apiVersion":"v1"}
machine created: test-mhc-machine-wqwzx
I0729 08:27:01.588648   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:27:01.595629   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-vrrqw\" not found" "name"="test-mhc-machine-vrrqw" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.595924   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-z7v7n" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-kkr8b"}
I0729 08:27:01.646603   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:27:01.678908   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-z7v7n\" not found" "name"="test-mhc-machine-z7v7n" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.695826   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.704626   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.733933   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
node created: test-mhc-node-h7wc7
I0729 08:27:01.757672   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.766241   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-g2zjk
I0729 08:27:01.782443   13091 controller.go:130] controller-runtime/manager/controller/machine "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "source"={}
machine created: test-mhc-machine-rmf2z
I0729 08:27:01.835383   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test6-lrz6f" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"id-node-1","uid":"39025841-0a60-407e-9ca9-925d79716290","apiVersion":"v1"}
E0729 08:27:01.836569   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="failed to get /, Kind= \"\" for Machine \"test6-v6pw9\" in namespace \"default\": failed to retrieve  external object \"default\"/\"\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:01.884384   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:01.901098   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.943746   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.954391   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.969691   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.969997   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
node created: test-mhc-node-hqvtc
I0729 08:27:02.022963   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.046407   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.053046   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.063850   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.075978   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-tb4cc" "machine"="test-mhc-machine-rmf2z" "name"="test-mhc-machine-rmf2z" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-hqvtc"
I0729 08:27:02.099386   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.111360   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
Cleaning up nodes, machines and infra machines.
I0729 08:27:02.172927   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:02.175525   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-tb4cc" "name"="test-mhc-9pb9t" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-gpxtw/test-mhc-9pb9t/test-mhc-machine-wqwzx/"
E0729 08:27:02.241464   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"test-mhc-machine-wqwzx\" in namespace \"test-mhc-gpxtw\": cannot find node with matching ProviderID" "name"="test-mhc-machine-wqwzx" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:02.241873   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-tb4cc" "name"="test-mhc-machine-rmf2z" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="no control plane members" "node"={"kind":"Node","name":"test-mhc-node-hqvtc","uid":"a591e025-dd49-462a-817a-3738f82e577b","apiVersion":"v1"}
I0729 08:27:02.247841   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=2
I0729 08:27:02.247910   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-rmf2z,test-mhc-machine-wqwzx" "indirect descendants count"=0
I0729 08:27:02.261896   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=2
I0729 08:27:02.261981   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-tb4cc" "namespace"="test-mhc-gpxtw" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-rmf2z,test-mhc-machine-wqwzx" "indirect descendants count"=0

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_ignores_Machines_not_matching_the_label_selector 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_ignores\_Machines\_not\_matching\_the\_label\_selector$'
=== RUN   TestMachineHealthCheck_Reconcile/it_ignores_Machines_not_matching_the_label_selector
E0729 08:26:59.425163   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-hp4hk\" not found" "name"="test-cluster-hp4hk" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:59.486436   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-vn275
I0729 08:26:59.656601   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:59.855084   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.867868   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.867940   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.868823   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.869831   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
machine created: test-mhc-machine-vrrqw
I0729 08:26:59.903855   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.914394   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.929909   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.944222   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.963007   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.976993   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:59.977277   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
node created: test-mhc-node-rj8zn
I0729 08:26:59.985683   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-q2pg2" "machine"="test-mhc-machine-vrrqw" "name"="test-mhc-machine-vrrqw" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-rj8zn"
I0729 08:27:00.009679   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
inframachine created: test-mhc-machine-infra-x4xc9
machine created: test-mhc-machine-z7v7n
I0729 08:27:00.081121   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.081550   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.131315   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.131971   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.248544   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.249132   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.262505   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.263080   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.276167   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.276845   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.293166   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.293725   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.307942   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.308616   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.312450   13091 controller.go:130] controller-runtime/manager/controller/machine "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "source"={}
I0729 08:27:00.380885   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.381816   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.406408   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.407081   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
node created: test-mhc-node-kkr8b
I0729 08:27:00.435470   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.436354   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="Node failed to report startup in &Duration{Duration:1ms,}" "reason"="NodeStartupTimeout" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:00.442561   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:27:00.455748   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
inframachine created: test-mhc-machine-infra-4vqrx
I0729 08:27:00.495808   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
machine created: test-mhc-machine-c5pwz
E0729 08:27:00.496183   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:00.524493   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test6-lrz6f" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"id-node-1","uid":"39025841-0a60-407e-9ca9-925d79716290","apiVersion":"v1"}
E0729 08:27:00.525847   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="failed to get /, Kind= \"\" for Machine \"test6-v6pw9\" in namespace \"default\": failed to retrieve  external object \"default\"/\"\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test6-v6pw9" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:00.604324   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.640400   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:00.662473   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
node created: test-mhc-node-nrbfg
I0729 08:27:00.811985   13091 machine_controller_noderef.go:89] controller-runtime/manager/controller/machine "msg"="Set Machine's NodeRef" "cluster"="test-cluster-q2pg2" "machine"="test-mhc-machine-c5pwz" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "noderef"="test-mhc-node-nrbfg"
inframachine created: test-mhc-machine-infra-lwx2t
machine created: test-mhc-machine-stq22
I0729 08:27:00.909350   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:00.909426   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
node created: test-mhc-node-hszf7
Cleaning up nodes, machines and infra machines.
I0729 08:27:01.187187   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.187251   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:27:01.204743   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.205887   13091 machine_controller.go:302] controller-runtime/manager/controller/machine "msg"="Draining node" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "node"="test-mhc-node-nrbfg"
Cleaning up nodes, machines and infra machines.
I0729 08:27:01.246778   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.247408   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-vrrqw/"
E0729 08:27:01.252087   13091 machine_controller.go:488] controller-runtime/manager/controller/machine "msg"="Could not find node from noderef, it may have already been deleted" "error"="nodes \"test-mhc-node-nrbfg\" not found" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "node"="test-mhc-node-nrbfg" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.288940   13091 machine_controller.go:359] controller-runtime/manager/controller/machine "msg"="Deleting node" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "node"="test-mhc-node-nrbfg"
I0729 08:27:01.303053   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.303762   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-vrrqw/"
I0729 08:27:01.304384   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
I0729 08:27:01.338320   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=4
I0729 08:27:01.338390   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test-cluster-q2pg2" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test-mhc-machine-vrrqw,test-mhc-machine-z7v7n;Worker machines: test-mhc-machine-stq22,test-mhc-machine-c5pwz" "indirect descendants count"=0
I0729 08:27:01.350248   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:27:01.350940   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-vrrqw/"
I0729 08:27:01.351514   13091 machinehealthcheck_controller.go:406] controller-runtime/manager/controller/machinehealthcheck "msg"="Target has failed health check, marking for remediation" "cluster"="test-cluster-q2pg2" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "message"="" "reason"="NodeNotFound" "target"="test-mhc-zgbvh/test-mhc-87v6c/test-mhc-machine-z7v7n/"
E0729 08:27:01.352872   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-c5pwz\" not found" "name"="test-mhc-machine-c5pwz" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:27:01.353096   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-q2pg2" "name"="test-mhc-machine-stq22" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-hszf7"}
I0729 08:27:01.358913   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-87v6c" "namespace"="test-mhc-zgbvh" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_modifying_existing_ones 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_should\_ensure\_an\_owner\_reference\_is\_present\_when\_modifying\_existing\_ones$'
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_modifying_existing_ones
I0729 08:26:59.041965   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:59.057713   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:26:59.109690   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-tltwd\" not found" "name"="test-cluster-tltwd" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:59.152110   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.286417   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
I0729 08:26:59.318817   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.332634   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.372995   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-f7mwn" "namespace"="test-mhc-hj9l5" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_no_existing_ones_exist 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_should\_ensure\_an\_owner\_reference\_is\_present\_when\_no\_existing\_ones\_exist$'
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_no_existing_ones_exist
E0729 08:26:58.642798   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-jk6g6\" not found" "name"="test-cluster-jk6g6" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:58.710849   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.712190   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-jk6g6\" not found"  "cluster"="test-mhc-274lr/test-cluster-jk6g6"
E0729 08:26:58.720930   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-4z9jq\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-4z9jq\" not found]" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.721151   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-5kkmb" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.721535   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.723105   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.723629   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster" "name"="md-57h7g-5575b6c97-hkppq" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"md-57h7g-5575b6c97-hkppq-9qrsc","uid":"6fadd2e2-060b-4360-b8da-7e13f2fc76bf","apiVersion":"v1"}
I0729 08:26:58.726264   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.781200   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:58.781414   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.799884   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.802435   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97-hkppq" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
E0729 08:26:58.802808   13091 machineset_controller.go:410] controller-runtime/manager/controller/machineset "msg"="Unable to delete Machine" "error"="machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.806571   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359287edcff7", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"FailedDelete", Message:"Failed to delete machine \"md-57h7g-5575b6c97-hkppq\": machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774afdd1bf7, ext:31505004080, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774afdd1bf7, ext:31505004080, loc:(*time.Location)(0x2bdcb20)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359287edcff7" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.827096   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.827157   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-bh8pt" "name"="md-57h7g-65496964d6-bh8pt" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.852180   13091 cluster_controller.go:231] controller-runtime/manager/controller/cluster "msg"="Cluster still has children - deleting them first" "name"="test6-lrz6f" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "count"=1
I0729 08:26:58.852309   13091 cluster_controller.go:257] controller-runtime/manager/controller/cluster "msg"="Cluster still has descendants - need to requeue" "name"="test6-lrz6f" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" "descendants"="Control plane machines: test6-v6pw9" "indirect descendants count"=0
I0729 08:26:58.932155   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.933072   13091 machineset_controller.go:159] controller-runtime/manager/controller/machineset "msg"="Failed to reconcile MachineSet" "error"="failed to sync MachineSet replicas: machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" 
E0729 08:26:58.933508   13091 controller.go:304] controller-runtime/manager/controller/machineset "msg"="Reconciler error" "error"="failed to sync MachineSet replicas: machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" 
E0729 08:26:58.936721   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.169635928fae9594", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"ReconcileError", Message:"failed to sync MachineSet replicas: machines.cluster.x-k8s.io \"md-57h7g-5575b6c97-hkppq\" not found", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774b79de194, ext:31635078088, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774b79de194, ext:31635078088, loc:(*time.Location)(0x2bdcb20)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.169635928fae9594" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.961619   13091 machine_controller_phases.go:282] controller-runtime/manager/controller/machine "msg"="Infrastructure provider is not ready, requeuing" "cluster"="test-cluster" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.961692   13091 machine_controller_noderef.go:47] controller-runtime/manager/controller/machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "cluster"="test-cluster" "machine"="md-57h7g-65496964d6-9xkgf" "name"="md-57h7g-65496964d6-9xkgf" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.974656   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:59.006285   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-l4zkt" "namespace"="test-mhc-h2ts9" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_no_existing_labels_exist 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_should\_ensure\_the\_correct\_cluster\-name\_label\_when\_no\_existing\_labels\_exist$'
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_no_existing_labels_exist
E0729 08:26:57.976502   13091 controller.go:304] controller-runtime/manager/controller/machinedeployment "msg"="Reconciler error" "error"="machinedeployments.cluster.x-k8s.io \"md-57h7g\" not found" "name"="md-57h7g" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineDeployment" 
I0729 08:26:57.999117   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:57.999446   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.021017   13091 machineset_controller.go:415] controller-runtime/manager/controller/machineset "msg"="Deleted machine" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.027599   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359255ef5cf3", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"SuccessfulDelete", Message:"Deleted machine \"md-57h7g-5575b6c97-hkppq\"", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774797972f3, ext:30666244908, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774814159f6, ext:30723044432, loc:(*time.Location)(0x2bdcb20)}}, Count:2, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359255ef5cf3" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.043129   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_other_labels_are_present 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_should\_ensure\_the\_correct\_cluster\-name\_label\_when\_other\_labels\_are\_present$'
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_other_labels_are_present
E0729 08:26:58.333059   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-8bqsk\" not found" "name"="test-cluster-8bqsk" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
E0729 08:26:58.370768   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-9bkkk: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test4-9bkkk" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
E0729 08:26:58.402899   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-8bqsk\" not found"  "cluster"="test-mhc-vwrcx/test-cluster-8bqsk"
I0729 08:26:58.423344   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:58.423403   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.429786   13091 machineset_controller.go:415] controller-runtime/manager/controller/machineset "msg"="Deleted machine" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.435079   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359255ef5cf3", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"SuccessfulDelete", Message:"Deleted machine \"md-57h7g-5575b6c97-hkppq\"", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774797972f3, ext:30666244908, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774999ec101, ext:31131818820, loc:(*time.Location)(0x2bdcb20)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359255ef5cf3" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)
I0729 08:26:58.529542   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.531069   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-8bqsk\" not found"  "cluster"="test-mhc-vwrcx/test-cluster-8bqsk"
I0729 08:26:58.549133   13091 machineset_controller.go:398] controller-runtime/manager/controller/machineset "msg"="Too many replicas" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "deleting"=1 "need"=0
I0729 08:26:58.549209   13091 machineset_controller.go:404] controller-runtime/manager/controller/machineset "msg"="Found delete policy" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "delete-policy"="Newest"
I0729 08:26:58.558459   13091 machineset_controller.go:415] controller-runtime/manager/controller/machineset "msg"="Deleted machine" "name"="md-57h7g-5575b6c97" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineSet" "machine"="md-57h7g-5575b6c97-hkppq"
E0729 08:26:58.561883   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-5kkmb\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-5kkmb\" not found]" "name"="test-mhc-5kkmb" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.561971   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-4z9jq" "namespace"="test-mhc-274lr" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.573022   13091 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"md-57h7g-5575b6c97.1696359255ef5cf3", GenerateName:"", Namespace:"md-test", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"MachineSet", Namespace:"md-test", Name:"md-57h7g-5575b6c97", UID:"be1811fd-f18b-4673-9deb-1fc07043db3f", APIVersion:"cluster.x-k8s.io/v1alpha4", ResourceVersion:"930", FieldPath:""}, Reason:"SuccessfulDelete", Message:"Deleted machine \"md-57h7g-5575b6c97-hkppq\"", Source:v1.EventSource{Component:"machineset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc038b774797972f3, ext:30666244908, loc:(*time.Location)(0x2bdcb20)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc038b774a14a25b4, ext:31260491756, loc:(*time.Location)(0x2bdcb20)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events "md-57h7g-5575b6c97.1696359255ef5cf3" is forbidden: unable to create new content in namespace md-test because it is being terminated' (will not retry!)

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_the_label_has_the_wrong_value 0.00s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\sTestMachineHealthCheck\_Reconcile\/it\_should\_ensure\_the\_correct\_cluster\-name\_label\_when\_the\_label\_has\_the\_wrong\_value$'
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_the_label_has_the_wrong_value
E0729 08:26:58.112529   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-rn42k\" not found" "name"="test-cluster-rn42k" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
I0729 08:26:58.171961   13091 controller.go:130] controller-runtime/manager/controller/machinehealthcheck "msg"="Starting EventSource" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" "source"={}
E0729 08:26:58.173009   13091 deleg.go:144]  "msg"="Error health checking cluster" "error"="Cluster.cluster.x-k8s.io \"test-cluster-rn42k\" not found"  "cluster"="test-mhc-jsszl/test-cluster-rn42k"
E0729 08:26:58.190482   13091 controller.go:304] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-rrjb2\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-rrjb2\" not found]" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.190551   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
I0729 08:26:58.195849   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-rrjb2" "namespace"="test-mhc-jsszl" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.226851   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-kf4h4\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-kf4h4" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.268899   13091 machinehealthcheck_controller.go:108] controller-runtime/manager/controller/machinehealthcheck "msg"="Reconciling" "name"="test-mhc-5kkmb" "namespace"="test-mhc-vwrcx" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="MachineHealthCheck" 
E0729 08:26:58.277367   13091 controller.go:304] controller-runtime/manager/controller/cluster "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-vc7jp: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "name"="test2-vc7jp" "namespace"="default" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Cluster" 
E0729 08:26:58.297661   13091 controller.go:304] controller-runtime/manager/controller/machine "msg"="Reconciler error" "error"="no matching Node for Machine \"md-57h7g-684b9cfc89-9znlc\" in namespace \"md-test\": cannot find node with matching ProviderID" "name"="md-57h7g-684b9cfc89-9znlc" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" 
I0729 08:26:58.297982   13091 machine_controller.go:280] controller-runtime/manager/controller/machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster" "name"="md-57h7g-5575b6c97-hkppq" "namespace"="md-test" "reconciler group"="cluster.x-k8s.io" "reconciler kind"="Machine" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"md-57h7g-5575b6c97-hkppq-9qrsc","uid":"6fadd2e2-060b-4360-b8da-7e13f2fc76bf","apiVersion":"v1"}

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers/remote TestClusterCacheHealthCheck 5.05s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\/remote\sTestClusterCacheHealthCheck$'
=== RUN   TestClusterCacheHealthCheck
--- FAIL: TestClusterCacheHealthCheck (5.05s)

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers/remote TestClusterCacheHealthCheck/when_health_checking_clusters 5.05s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\/remote\sTestClusterCacheHealthCheck\/when\_health\_checking\_clusters$'
=== RUN   TestClusterCacheHealthCheck/when_health_checking_clusters
    --- FAIL: TestClusterCacheHealthCheck/when_health_checking_clusters (5.05s)

				from junit.xml

Filter through log files | View test history on testgrid


sigs.k8s.io/cluster-api/controllers/remote TestClusterCacheHealthCheck/when_health_checking_clusters/with_a_healthy_cluster 3.97s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=sigs\.k8s\.io\/cluster\-api\/controllers\/remote\sTestClusterCacheHealthCheck\/when\_health\_checking\_clusters\/with\_a\_healthy\_cluster$'
=== RUN   TestClusterCacheHealthCheck/when_health_checking_clusters/with_a_healthy_cluster
    cluster_cache_healthcheck_test.go:57: Setting up a new manager
    cluster_cache_healthcheck_test.go:66: Starting the manager
    cluster_cache_healthcheck_test.go:74: Setting up a ClusterCacheTracker
    cluster_cache_healthcheck_test.go:87: Creating a namespace for the test
    cluster_cache_healthcheck_test.go:91: Creating a test cluster
    cluster_cache_healthcheck_test.go:103: Creating a test cluster kubeconfig
E0729 08:26:49.545248   13327 cluster_cache.go:399]  "msg"="Error health checking cluster" "error"="Get \"https://127.0.0.1:42581/?timeout=50ms\": context deadline exceeded"  "cluster"="cluster-cache-test-gjnkv/test-cluster"
    cluster_cache_healthcheck_test.go:142: 
        Failed after 1.534s.
        Expected
            <bool>: false
        to be true
    cluster_cache_healthcheck_test.go:114: Deleting any Secrets
    cluster_cache_healthcheck_test.go:116: Deleting any Clusters
    cluster_cache_healthcheck_test.go:118: Stopping the manager
        --- FAIL: TestClusterCacheHealthCheck/when_health_checking_clusters/with_a_healthy_cluster (3.97s)

				from junit.xml

Filter through log files | View test history on testgrid


Show 1611 Passed Tests