ci-cri-containerd-e2e-cos-gce-proto #1595641597532311552
Recent runs || View in Spyglass
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAPI\spriority\sand\sfairness\sshould\sensure\sthat\srequests\scan\'t\sbe\sdrowned\sout\s\(fairness\)$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000f82f00) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] API priority and fairness set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:30:20.121 Nov 24 05:30:20.121: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename apf 11/24/22 05:30:20.123 Nov 24 05:30:20.162: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:22.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:24.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:26.202: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:28.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:30.202: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:32.202: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:34.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:36.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:38.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:40.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:42.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:44.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:46.203: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:48.202: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:50.202: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:50.241: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:50.241: INFO: Unexpected error: <*errors.errorString | 0xc00017fa50>: { s: "timed out waiting for the condition", } Nov 24 05:30:50.241: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000f82f00) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] API priority and fairness test/e2e/framework/node/init/init.go:32 Nov 24 05:30:50.242: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] API priority and fairness dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:30:50.281 [DeferCleanup (Each)] [sig-api-machinery] API priority and fairness tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAPI\spriority\sand\sfairness\sshould\sensure\sthat\srequests\scan\'t\sbe\sdrowned\sout\s\(priority\)$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010f7ef0) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] API priority and fairness set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:33:21.111 Nov 24 05:33:21.111: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename apf 11/24/22 05:33:21.113 Nov 24 05:33:21.152: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:23.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:25.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:27.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:29.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:31.191: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:33.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:35.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:37.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:39.193: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:41.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:43.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:45.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:47.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:49.192: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:51.191: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:51.231: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:33:51.231: INFO: Unexpected error: <*errors.errorString | 0xc0000d1db0>: { s: "timed out waiting for the condition", } Nov 24 05:33:51.231: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010f7ef0) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] API priority and fairness test/e2e/framework/node/init/init.go:32 Nov 24 05:33:51.231: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] API priority and fairness dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:33:51.271 [DeferCleanup (Each)] [sig-api-machinery] API priority and fairness tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\slisting\smutating\swebhooks\sshould\swork\s\[Conformance\]$'
test/e2e/apimachinery/webhook.go:683 k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.19() test/e2e/apimachinery/webhook.go:683 +0x5a8from junit_01.xml
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:28:12.956 Nov 24 05:28:12.956: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename webhook 11/24/22 05:28:12.958 STEP: Waiting for a default service account to be provisioned in namespace 11/24/22 05:28:13.222 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/24/22 05:28:13.328 [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/apimachinery/webhook.go:90 STEP: Setting up server cert 11/24/22 05:28:13.637 STEP: Create role binding to let webhook read extension-apiserver-authentication 11/24/22 05:28:14.164 STEP: Deploying the webhook pod 11/24/22 05:28:14.566 STEP: Wait for the deployment to be ready 11/24/22 05:28:15.243 Nov 24 05:28:15.667: INFO: deployment "sample-webhook-deployment" doesn't have the required revision set Nov 24 05:28:17.840: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 16, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:28:20.283: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 16, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:28:21.956: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 16, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 15, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} STEP: Deploying the webhook service 11/24/22 05:28:24.013 STEP: Verifying the service has paired with the endpoint 11/24/22 05:28:24.304 Nov 24 05:28:25.306: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1 [It] listing mutating webhooks should work [Conformance] test/e2e/apimachinery/webhook.go:656 STEP: Listing all of the created validation webhooks 11/24/22 05:28:26.614 Nov 24 05:28:27.183: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:27.974: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:28.405: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:29.060: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:30.860: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:32.094: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:32.621: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:33.190: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:33.733: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:34.407: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:36.295: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:56.717: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:57.232: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:58.120: INFO: Waiting for webhook configuration to be ready... Nov 24 05:28:58.120: INFO: Unexpected error: waiting for webhook configuration to be ready: <*errors.errorString | 0xc0001cb910>: { s: "timed out waiting for the condition", } Nov 24 05:28:58.120: FAIL: waiting for webhook configuration to be ready: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.19() test/e2e/apimachinery/webhook.go:683 +0x5a8 [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:28:58.121: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/apimachinery/webhook.go:105 [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:28:58.536 STEP: Collecting events from namespace "webhook-6113". 11/24/22 05:28:58.536 STEP: Found 9 events. 11/24/22 05:28:58.599 Nov 24 05:28:58.599: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for sample-webhook-deployment-865554f4d9-j7w6m: { } Scheduled: Successfully assigned webhook-6113/sample-webhook-deployment-865554f4d9-j7w6m to bootstrap-e2e-minion-group-gz1x Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:15 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-865554f4d9 to 1 Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:15 +0000 UTC - event for sample-webhook-deployment-865554f4d9: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-865554f4d9-j7w6m Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:17 +0000 UTC - event for sample-webhook-deployment-865554f4d9-j7w6m: {kubelet bootstrap-e2e-minion-group-gz1x} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:17 +0000 UTC - event for sample-webhook-deployment-865554f4d9-j7w6m: {kubelet bootstrap-e2e-minion-group-gz1x} Created: Created container sample-webhook Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:17 +0000 UTC - event for sample-webhook-deployment-865554f4d9-j7w6m: {kubelet bootstrap-e2e-minion-group-gz1x} Started: Started container sample-webhook Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:18 +0000 UTC - event for sample-webhook-deployment-865554f4d9-j7w6m: {kubelet bootstrap-e2e-minion-group-gz1x} Killing: Stopping container sample-webhook Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:19 +0000 UTC - event for sample-webhook-deployment-865554f4d9-j7w6m: {kubelet bootstrap-e2e-minion-group-gz1x} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Nov 24 05:28:58.599: INFO: At 2022-11-24 05:28:19 +0000 UTC - event for sample-webhook-deployment-865554f4d9-j7w6m: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.226:8444/readyz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Nov 24 05:28:58.655: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 05:28:58.655: INFO: Nov 24 05:28:58.708: INFO: Logging node info for node bootstrap-e2e-master Nov 24 05:28:58.822: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 15023 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:28:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:28:58.823: INFO: Logging kubelet events for node bootstrap-e2e-master Nov 24 05:28:58.995: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Nov 24 05:28:59.220: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available Nov 24 05:28:59.220: INFO: Logging node info for node bootstrap-e2e-minion-group-gz1x Nov 24 05:28:59.431: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 15563 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8250":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8355":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:27:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:28:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:28:59.432: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-gz1x Nov 24 05:28:59.575: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x Nov 24 05:28:59.749: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-gz1x: error trying to reach service: No agent available Nov 24 05:28:59.750: INFO: Logging node info for node bootstrap-e2e-minion-group-phcp Nov 24 05:28:59.847: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 16124 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-8063":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-24 05:27:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:28:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},} Nov 24 05:28:59.847: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-phcp Nov 24 05:29:00.170: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp Nov 24 05:29:00.318: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-phcp: error trying to reach service: No agent available Nov 24 05:29:00.318: INFO: Logging node info for node bootstrap-e2e-minion-group-qvgt Nov 24 05:29:00.431: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 16025 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:28:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:28:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-1981^db0f04ed-6bb8-11ed-8716-be3755f74117],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-1981^db0f04ed-6bb8-11ed-8716-be3755f74117,DevicePath:,},},Config:nil,},} Nov 24 05:29:00.432: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qvgt Nov 24 05:29:00.669: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt Nov 24 05:29:00.865: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-qvgt: error trying to reach service: No agent available STEP: Collecting events from namespace "webhook-6113-markers". 11/24/22 05:29:00.865 STEP: Found 0 events. 11/24/22 05:29:00.972 Nov 24 05:29:01.096: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 05:29:01.096: INFO: Nov 24 05:29:01.160: INFO: Logging node info for node bootstrap-e2e-master Nov 24 05:29:01.224: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 15023 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:28:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:29:01.224: INFO: Logging kubelet events for node bootstrap-e2e-master Nov 24 05:29:01.329: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Nov 24 05:29:01.434: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available Nov 24 05:29:01.434: INFO: Logging node info for node bootstrap-e2e-minion-group-gz1x Nov 24 05:29:01.508: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 15563 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8250":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8355":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:27:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:28:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:29:01.509: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-gz1x Nov 24 05:29:01.585: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x Nov 24 05:29:01.684: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-gz1x: error trying to reach service: No agent available Nov 24 05:29:01.684: INFO: Logging node info for node bootstrap-e2e-minion-group-phcp Nov 24 05:29:01.748: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 16124 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-8063":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-24 05:27:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:28:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},} Nov 24 05:29:01.748: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-phcp Nov 24 05:29:01.803: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp Nov 24 05:29:01.970: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-phcp: error trying to reach service: No agent available Nov 24 05:29:01.970: INFO: Logging node info for node bootstrap-e2e-minion-group-qvgt Nov 24 05:29:02.098: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 16025 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:28:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:28:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:46 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-1981^db0f04ed-6bb8-11ed-8716-be3755f74117],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-1981^db0f04ed-6bb8-11ed-8716-be3755f74117,DevicePath:,},},Config:nil,},} Nov 24 05:29:02.099: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qvgt Nov 24 05:29:02.191: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt Nov 24 05:29:02.306: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-qvgt: error trying to reach service: No agent available [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] tear down framework | framework.go:193 STEP: Destroying namespace "webhook-6113" for this suite. 11/24/22 05:29:02.306 STEP: Destroying namespace "webhook-6113-markers" for this suite. 11/24/22 05:29:02.377
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\spatching\/updating\sa\smutating\swebhook\sshould\swork\s\[Conformance\]$'
test/e2e/apimachinery/webhook.go:853
k8s.io/kubernetes/test/e2e/apimachinery.deployWebhookAndService(0xc0003a3950, {0xc004992090, 0x2c}, 0xc0048bc0f0, 0x20fb, 0x20fc)
test/e2e/apimachinery/webhook.go:853 +0x13da
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.1()
test/e2e/apimachinery/webhook.go:102 +0x226
There were additional failures detected after the initial failure:
[FAILED]
Nov 24 05:29:49.804: Couldn't delete ns: "webhook-2373": Delete "https://34.83.136.178/api/v1/namespaces/webhook-2373": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-2373", Err:(*net.OpError)(0xc002622690)}),Couldn't delete ns: "webhook-2373-markers": Delete "https://34.83.136.178/api/v1/namespaces/webhook-2373-markers": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-2373-markers", Err:(*net.OpError)(0xc00162d950)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:29:02.347
Nov 24 05:29:02.347: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:29:02.349
STEP: Waiting for a default service account to be provisioned in namespace 11/24/22 05:29:02.525
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/24/22 05:29:02.618
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:90
STEP: Setting up server cert 11/24/22 05:29:02.969
STEP: Create role binding to let webhook read extension-apiserver-authentication 11/24/22 05:29:03.969
STEP: Deploying the webhook pod 11/24/22 05:29:04.134
STEP: Wait for the deployment to be ready 11/24/22 05:29:04.368
Nov 24 05:29:04.587: INFO: deployment "sample-webhook-deployment" doesn't have the required revision set
Nov 24 05:29:06.786: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:08.881: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:10.852: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:12.884: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:14.902: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 29, 4, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
STEP: Deploying the webhook service 11/24/22 05:29:16.872
STEP: Verifying the service has paired with the endpoint 11/24/22 05:29:16.971
Nov 24 05:29:17.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:18.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:19.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:20.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:21.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:22.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:23.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:24.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:25.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:26.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:27.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:28.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:29.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:30.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:31.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:32.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:33.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:34.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:35.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:36.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:37.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:38.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:39.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:40.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:41.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:42.971: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:43.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:44.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:45.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:46.972: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:47.020: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 24 05:29:47.073: INFO: Unexpected error: waiting for service webhook-2373/e2e-test-webhook have 1 endpoint:
<*errors.errorString | 0xc0000d1db0>: {
s: "timed out waiting for the condition",
}
Nov 24 05:29:47.073: FAIL: waiting for service webhook-2373/e2e-test-webhook have 1 endpoint: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.deployWebhookAndService(0xc0003a3950, {0xc004992090, 0x2c}, 0xc0048bc0f0, 0x20fb, 0x20fc)
test/e2e/apimachinery/webhook.go:853 +0x13da
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.1()
test/e2e/apimachinery/webhook.go:102 +0x226
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:29:47.073: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:29:47.545
STEP: Collecting events from namespace "webhook-2373". 11/24/22 05:29:47.545
STEP: Found 14 events. 11/24/22 05:29:47.656
Nov 24 05:29:47.656: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: { } Scheduled: Successfully assigned webhook-2373/sample-webhook-deployment-865554f4d9-9cptl to bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:04 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-865554f4d9 to 1
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:04 +0000 UTC - event for sample-webhook-deployment-865554f4d9: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-865554f4d9-9cptl
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:08 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:08 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Created: Created container sample-webhook
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:08 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Started: Started container sample-webhook
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:10 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Killing: Stopping container sample-webhook
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:10 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.3:8444/readyz": dial tcp 10.64.1.3:8444: connect: connection refused
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:11 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:13 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.7:8444/readyz": dial tcp 10.64.1.7:8444: connect: connection refused
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:13 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.7:8444/readyz": EOF
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:15 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} BackOff: Back-off restarting failed container sample-webhook in pod sample-webhook-deployment-865554f4d9-9cptl_webhook-2373(9c5b5b72-e9cb-498a-8407-33de24852679)
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:28 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.14:8444/readyz": EOF
Nov 24 05:29:47.656: INFO: At 2022-11-24 05:29:28 +0000 UTC - event for sample-webhook-deployment-865554f4d9-9cptl: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.14:8444/readyz": dial tcp 10.64.1.14:8444: connect: connection refused
Nov 24 05:29:47.778: INFO: POD NODE PHASE GRACE CONDITIONS
Nov 24 05:29:47.778: INFO:
Nov 24 05:29:47.846: INFO:
Logging node info for node bootstrap-e2e-master
Nov 24 05:29:47.934: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 15023 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:28:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:47.935: INFO:
Logging kubelet events for node bootstrap-e2e-master
Nov 24 05:29:48.005: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 24 05:29:48.187: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 24 05:29:48.187: INFO:
Logging node info for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:48.248: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 17247 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-3519":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8250":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:27:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:29:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:48.248: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:48.306: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:48.376: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-gz1x: error trying to reach service: No agent available
Nov 24 05:29:48.376: INFO:
Logging node info for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:48.428: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 16972 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-4867":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-8063":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-24 05:27:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:29:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},}
Nov 24 05:29:48.428: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:48.486: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:48.593: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-phcp: error trying to reach service: No agent available
Nov 24 05:29:48.593: INFO:
Logging node info for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:48.645: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 16923 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:28:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-24 05:29:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:48.645: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:48.722: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:48.833: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-qvgt: error trying to reach service: No agent available
STEP: Collecting events from namespace "webhook-2373-markers". 11/24/22 05:29:48.834
STEP: Found 0 events. 11/24/22 05:29:48.928
Nov 24 05:29:48.988: INFO: POD NODE PHASE GRACE CONDITIONS
Nov 24 05:29:48.988: INFO:
Nov 24 05:29:49.051: INFO:
Logging node info for node bootstrap-e2e-master
Nov 24 05:29:49.173: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 15023 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:28:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.173: INFO:
Logging kubelet events for node bootstrap-e2e-master
Nov 24 05:29:49.244: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 24 05:29:49.372: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 24 05:29:49.372: INFO:
Logging node info for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:49.411: INFO: Error getting node info Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-gz1x": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.411: INFO: Node Info: &Node{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{},Allocatable:ResourceList{},Phase:,Conditions:[]NodeCondition{},Addresses:[]NodeAddress{},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:0,},},NodeInfo:NodeSystemInfo{MachineID:,SystemUUID:,BootID:,KernelVersion:,OSImage:,ContainerRuntimeVersion:,KubeletVersion:,KubeProxyVersion:,OperatingSystem:,Architecture:,},Images:[]ContainerImage{},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.412: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:49.451: INFO: Unexpected error retrieving node events Get "https://34.83.136.178/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.kind%3DNode%2CinvolvedObject.namespace%3D%2Csource%3Dkubelet%2CinvolvedObject.name%3Dbootstrap-e2e-minion-group-gz1x": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.451: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:49.490: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-gz1x: Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-gz1x:10250/proxy/pods": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.490: INFO:
Logging node info for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:49.529: INFO: Error getting node info Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-phcp": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.529: INFO: Node Info: &Node{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{},Allocatable:ResourceList{},Phase:,Conditions:[]NodeCondition{},Addresses:[]NodeAddress{},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:0,},},NodeInfo:NodeSystemInfo{MachineID:,SystemUUID:,BootID:,KernelVersion:,OSImage:,ContainerRuntimeVersion:,KubeletVersion:,KubeProxyVersion:,OperatingSystem:,Architecture:,},Images:[]ContainerImage{},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.529: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:49.568: INFO: Unexpected error retrieving node events Get "https://34.83.136.178/api/v1/namespaces/kube-system/events?fieldSelector=source%3Dkubelet%2CinvolvedObject.name%3Dbootstrap-e2e-minion-group-phcp%2CinvolvedObject.kind%3DNode%2CinvolvedObject.namespace%3D": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.568: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:49.607: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-phcp: Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-phcp:10250/proxy/pods": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.607: INFO:
Logging node info for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:49.646: INFO: Error getting node info Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-qvgt": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.646: INFO: Node Info: &Node{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{},Allocatable:ResourceList{},Phase:,Conditions:[]NodeCondition{},Addresses:[]NodeAddress{},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:0,},},NodeInfo:NodeSystemInfo{MachineID:,SystemUUID:,BootID:,KernelVersion:,OSImage:,ContainerRuntimeVersion:,KubeletVersion:,KubeProxyVersion:,OperatingSystem:,Architecture:,},Images:[]ContainerImage{},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.647: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:49.686: INFO: Unexpected error retrieving node events Get "https://34.83.136.178/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.kind%3DNode%2CinvolvedObject.namespace%3D%2Csource%3Dkubelet%2CinvolvedObject.name%3Dbootstrap-e2e-minion-group-qvgt": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.686: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:49.725: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-qvgt: Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-qvgt:10250/proxy/pods": dial tcp 34.83.136.178:443: connect: connection refused
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
STEP: Destroying namespace "webhook-2373" for this suite. 11/24/22 05:29:49.725
STEP: Destroying namespace "webhook-2373-markers" for this suite. 11/24/22 05:29:49.765
Nov 24 05:29:49.804: FAIL: Couldn't delete ns: "webhook-2373": Delete "https://34.83.136.178/api/v1/namespaces/webhook-2373": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-2373", Err:(*net.OpError)(0xc002622690)}),Couldn't delete ns: "webhook-2373-markers": Delete "https://34.83.136.178/api/v1/namespaces/webhook-2373-markers": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-2373-markers", Err:(*net.OpError)(0xc00162d950)})
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc0003a3950)
test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc0044af040?, 0xc004a16fb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc0044af040?, 0x0?}, {0xae73300?, 0x5?, 0xc000ffd170?})
/usr/local/go/src/reflect/value.go:368 +0xbc
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\spatching\/updating\sa\svalidating\swebhook\sshould\swork\s\[Conformance\]$'
test/e2e/apimachinery/webhook.go:435 k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.16() test/e2e/apimachinery/webhook.go:435 +0x454from junit_01.xml
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:03:11.657 Nov 24 05:03:11.657: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename webhook 11/24/22 05:03:11.659 Nov 24 05:03:11.698: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused STEP: Waiting for a default service account to be provisioned in namespace 11/24/22 05:04:42.261 STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/24/22 05:04:42.355 [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/framework/metrics/init/init.go:31 [BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/apimachinery/webhook.go:90 STEP: Setting up server cert 11/24/22 05:04:42.564 STEP: Create role binding to let webhook read extension-apiserver-authentication 11/24/22 05:04:43.108 STEP: Deploying the webhook pod 11/24/22 05:04:43.306 STEP: Wait for the deployment to be ready 11/24/22 05:04:43.649 Nov 24 05:04:44.015: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:0, UpdatedReplicas:0, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"NewReplicaSetCreated", Message:"Created new replica set \"sample-webhook-deployment-865554f4d9\""}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)} Nov 24 05:04:46.170: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:04:48.076: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:04:50.084: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:04:52.102: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:04:54.225: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:04:56.141: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:04:58.072: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} Nov 24 05:05:00.116: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 4, 44, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 4, 43, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)} STEP: Deploying the webhook service 11/24/22 05:05:02.094 STEP: Verifying the service has paired with the endpoint 11/24/22 05:05:02.249 Nov 24 05:05:03.250: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1 [It] patching/updating a validating webhook should work [Conformance] test/e2e/apimachinery/webhook.go:413 STEP: Creating a validating webhook configuration 11/24/22 05:05:03.306 Nov 24 05:05:03.600: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:03.897: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:04.071: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:04.371: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:04.760: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:05.049: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:05.266: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:06.563: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:06.837: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:08.093: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:09.564: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:10.038: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:11.355: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:11.546: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:11.886: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:13.143: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:14.451: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:14.763: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:15.062: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:16.349: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:16.525: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:16.825: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:17.042: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:17.230: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:17.498: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:17.731: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:17.990: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:18.165: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:18.327: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:18.542: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:18.734: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:18.947: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:19.186: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:19.606: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:19.895: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:20.139: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:20.377: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:20.623: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:20.904: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:21.562: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:22.040: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:22.451: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:22.732: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:23.022: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:23.288: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:23.583: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:23.815: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:24.349: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:25.637: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:25.887: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:26.042: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:26.253: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:26.408: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:26.652: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:26.814: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:27.031: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:27.283: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:27.450: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:27.623: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:27.855: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:28.062: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:28.235: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:28.435: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:28.837: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:29.199: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:29.458: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:29.666: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:29.845: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:30.070: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:30.264: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:30.459: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:30.687: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:30.866: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:31.080: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:31.226: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:31.436: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:31.617: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:31.839: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:32.044: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:32.223: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:32.425: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:32.617: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:32.865: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:33.099: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:33.297: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:33.459: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:33.636: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:33.818: INFO: Waiting for webhook configuration to be ready... Nov 24 05:05:33.818: INFO: Unexpected error: waiting for webhook configuration to be ready: <*errors.errorString | 0xc0001cb910>: { s: "timed out waiting for the condition", } Nov 24 05:05:33.818: FAIL: waiting for webhook configuration to be ready: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.16() test/e2e/apimachinery/webhook.go:435 +0x454 [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:05:33.910: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/apimachinery/webhook.go:105 [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/framework/metrics/init/init.go:33 [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:05:34.612 STEP: Collecting events from namespace "webhook-4642". 11/24/22 05:05:34.612 STEP: Found 13 events. 11/24/22 05:05:34.674 Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:43 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-865554f4d9 to 1 Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:43 +0000 UTC - event for sample-webhook-deployment-865554f4d9: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-865554f4d9-t8lqt Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:43 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {default-scheduler } Scheduled: Successfully assigned webhook-4642/sample-webhook-deployment-865554f4d9-t8lqt to bootstrap-e2e-minion-group-gz1x Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:45 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} FailedMount: MountVolume.SetUp failed for volume "webhook-certs" : failed to sync secret cache: timed out waiting for the condition Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:54 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:54 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} Created: Created container sample-webhook Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:54 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} Started: Started container sample-webhook Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:55 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.88:8444/readyz": dial tcp 10.64.1.88:8444: connect: connection refused Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:55 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} Killing: Stopping container sample-webhook Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:56 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} SandboxChanged: Pod sandbox changed, it will be killed and re-created. Nov 24 05:05:34.674: INFO: At 2022-11-24 05:04:59 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.94:8444/readyz": dial tcp 10.64.1.94:8444: connect: connection refused Nov 24 05:05:34.674: INFO: At 2022-11-24 05:05:03 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} BackOff: Back-off restarting failed container sample-webhook in pod sample-webhook-deployment-865554f4d9-t8lqt_webhook-4642(33f04653-54fc-4141-bd03-de71174db181) Nov 24 05:05:34.674: INFO: At 2022-11-24 05:05:18 +0000 UTC - event for sample-webhook-deployment-865554f4d9-t8lqt: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.102:8444/readyz": dial tcp 10.64.1.102:8444: connect: connection refused Nov 24 05:05:34.733: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 05:05:34.733: INFO: Nov 24 05:05:34.863: INFO: Logging node info for node bootstrap-e2e-master Nov 24 05:05:34.925: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 3828 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:04:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:05:34.926: INFO: Logging kubelet events for node bootstrap-e2e-master Nov 24 05:05:35.295: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Nov 24 05:05:35.562: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-24 04:57:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container kube-addon-manager ready: true, restart count 1 Nov 24 05:05:35.562: INFO: metadata-proxy-v0.1-ww2rj started at 2022-11-24 04:58:45 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:35.562: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:35.562: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:35.562: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container etcd-container ready: true, restart count 2 Nov 24 05:05:35.562: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 05:05:35.562: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container etcd-container ready: true, restart count 1 Nov 24 05:05:35.562: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container konnectivity-server-container ready: true, restart count 1 Nov 24 05:05:35.562: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-24 04:57:52 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container l7-lb-controller ready: true, restart count 4 Nov 24 05:05:35.562: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container kube-controller-manager ready: true, restart count 4 Nov 24 05:05:35.562: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:35.562: INFO: Container kube-scheduler ready: true, restart count 2 Nov 24 05:05:35.875: INFO: Latency metrics for node bootstrap-e2e-master Nov 24 05:05:35.875: INFO: Logging node info for node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:35.929: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 5082 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1793":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:03:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:04:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-24 05:05:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:05:35.930: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:36.021: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:36.269: INFO: test-pod-1 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container token-test ready: false, restart count 0 Nov 24 05:05:36.269: INFO: rs-k8v86 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:36.269: INFO: netserver-0 started at 2022-11-24 05:05:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container webserver ready: false, restart count 1 Nov 24 05:05:36.269: INFO: fail-once-local-hs44m started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container c ready: false, restart count 1 Nov 24 05:05:36.269: INFO: rs-ndmwk started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:36.269: INFO: netserver-0 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container webserver ready: true, restart count 0 Nov 24 05:05:36.269: INFO: csi-mockplugin-0 started at 2022-11-24 05:04:45 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:36.269: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:36.269: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 05:05:36.269: INFO: Container mock ready: true, restart count 0 Nov 24 05:05:36.269: INFO: verify-service-up-exec-pod-76q9m started at 2022-11-24 05:05:07 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:05:36.269: INFO: fail-once-local-2b8gh started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container c ready: false, restart count 1 Nov 24 05:05:36.269: INFO: logs-generator started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container logs-generator ready: false, restart count 0 Nov 24 05:05:36.269: INFO: hostpath-client started at 2022-11-24 05:05:10 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container hostpath-client ready: false, restart count 1 Nov 24 05:05:36.269: INFO: netserver-0 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container webserver ready: true, restart count 0 Nov 24 05:05:36.269: INFO: liveness-d2d53f99-b5a7-4102-9424-8e2aae6c8329 started at 2022-11-24 05:04:55 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container agnhost-container ready: true, restart count 2 Nov 24 05:05:36.269: INFO: metadata-proxy-v0.1-f4l4j started at 2022-11-24 04:58:19 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:36.269: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:36.269: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:36.269: INFO: hostexec-bootstrap-e2e-minion-group-gz1x-jcbgl started at 2022-11-24 05:02:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:05:36.269: INFO: pod-ready started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container pod-readiness-gate ready: true, restart count 0 Nov 24 05:05:36.269: INFO: pod-tk9n9 started at 2022-11-24 05:02:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container agnhost ready: true, restart count 1 Nov 24 05:05:36.269: INFO: rs-9czwp started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container donothing ready: true, restart count 0 Nov 24 05:05:36.269: INFO: test-webserver-551993bf-4ba0-4a93-92a1-0771237b1b1e started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container test-webserver ready: false, restart count 0 Nov 24 05:05:36.269: INFO: konnectivity-agent-4tkcz started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container konnectivity-agent ready: false, restart count 3 Nov 24 05:05:36.269: INFO: liveness-ecce2bb9-da3e-402f-8446-00acbacd1ce3 started at 2022-11-24 05:01:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container agnhost-container ready: true, restart count 3 Nov 24 05:05:36.269: INFO: test-grpc-1c6bd144-5b65-4cb4-91dc-a99a6f2d8d15 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container etcd ready: true, restart count 2 Nov 24 05:05:36.269: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:04:45 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:36.269: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:12 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:36.269: INFO: Container csi-attacher ready: false, restart count 4 Nov 24 05:05:36.269: INFO: Container csi-provisioner ready: false, restart count 4 Nov 24 05:05:36.269: INFO: Container csi-resizer ready: false, restart count 4 Nov 24 05:05:36.269: INFO: Container csi-snapshotter ready: false, restart count 4 Nov 24 05:05:36.269: INFO: Container hostpath ready: false, restart count 4 Nov 24 05:05:36.269: INFO: Container liveness-probe ready: false, restart count 4 Nov 24 05:05:36.269: INFO: Container node-driver-registrar ready: false, restart count 4 Nov 24 05:05:36.269: INFO: affinity-clusterip-transition-gqrb4 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container affinity-clusterip-transition ready: true, restart count 2 Nov 24 05:05:36.269: INFO: rs-fpj4b started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container donothing ready: false, restart count 2 Nov 24 05:05:36.269: INFO: netserver-0 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:36.269: INFO: kube-proxy-bootstrap-e2e-minion-group-gz1x started at 2022-11-24 04:58:18 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.269: INFO: Container kube-proxy ready: true, restart count 4 Nov 24 05:05:36.269: INFO: metrics-server-v0.5.2-867b8754b9-6fxbz started at 2022-11-24 04:59:43 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:36.269: INFO: Container metrics-server ready: false, restart count 3 Nov 24 05:05:36.269: INFO: Container metrics-server-nanny ready: false, restart count 4 Nov 24 05:05:36.270: INFO: pod-handle-http-request started at 2022-11-24 05:02:33 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:36.270: INFO: Container container-handle-http-request ready: true, restart count 2 Nov 24 05:05:36.270: INFO: Container container-handle-https-request ready: true, restart count 2 Nov 24 05:05:36.270: INFO: external-provisioner-tb7tr started at 2022-11-24 05:05:11 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.270: INFO: Container nfs-provisioner ready: false, restart count 1 Nov 24 05:05:36.639: INFO: Latency metrics for node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:36.639: INFO: Logging node info for node bootstrap-e2e-minion-group-phcp Nov 24 05:05:36.692: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 5036 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-3606":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-3727":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-24 05:03:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:05:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:05:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},},Config:nil,},} Nov 24 05:05:36.693: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-phcp Nov 24 05:05:36.759: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp Nov 24 05:05:36.992: INFO: coredns-6d97d5ddb-s4dqd started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container coredns ready: true, restart count 3 Nov 24 05:05:36.992: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:36.992: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:04:45 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-attacher ready: true, restart count 1 Nov 24 05:05:36.992: INFO: csi-mockplugin-0 started at 2022-11-24 05:02:32 +0000 UTC (0+4 container statuses recorded) Nov 24 05:05:36.992: INFO: Container busybox ready: true, restart count 3 Nov 24 05:05:36.992: INFO: Container csi-provisioner ready: false, restart count 3 Nov 24 05:05:36.992: INFO: Container driver-registrar ready: false, restart count 3 Nov 24 05:05:36.992: INFO: Container mock ready: false, restart count 3 Nov 24 05:05:36.992: INFO: rs-ppkb5 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container donothing ready: true, restart count 0 Nov 24 05:05:36.992: INFO: rs-6mxtz started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container donothing ready: false, restart count 2 Nov 24 05:05:36.992: INFO: netserver-1 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:36.992: INFO: konnectivity-agent-2t55k started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container konnectivity-agent ready: true, restart count 2 Nov 24 05:05:36.992: INFO: pod-handle-http-request started at 2022-11-24 05:05:03 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:36.992: INFO: Container container-handle-http-request ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container container-handle-https-request ready: true, restart count 0 Nov 24 05:05:36.992: INFO: pod-server-1 started at 2022-11-24 05:02:22 +0000 UTC (1+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Init container init ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container agnhost-container ready: true, restart count 1 Nov 24 05:05:36.992: INFO: netserver-1 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:36.992: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-provisioner ready: true, restart count 3 Nov 24 05:05:36.992: INFO: Container driver-registrar ready: true, restart count 3 Nov 24 05:05:36.992: INFO: Container mock ready: true, restart count 3 Nov 24 05:05:36.992: INFO: pod-with-prestop-https-hook started at 2022-11-24 05:05:09 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container pod-with-prestop-https-hook ready: false, restart count 2 Nov 24 05:05:36.992: INFO: metadata-proxy-v0.1-lghrl started at 2022-11-24 04:58:23 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:36.992: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:36.992: INFO: l7-default-backend-8549d69d99-29vxh started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container default-http-backend ready: true, restart count 0 Nov 24 05:05:36.992: INFO: volume-snapshot-controller-0 started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container volume-snapshot-controller ready: true, restart count 2 Nov 24 05:05:36.992: INFO: netserver-1 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container webserver ready: false, restart count 2 Nov 24 05:05:36.992: INFO: inline-volume-tester-htgjr started at 2022-11-24 05:02:36 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:05:36.992: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:04:49 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-attacher ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container csi-resizer ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container csi-snapshotter ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container hostpath ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container liveness-probe ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container node-driver-registrar ready: true, restart count 1 Nov 24 05:05:36.992: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:30 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:36.992: INFO: e2e-configmap-dns-server-30586570-454b-434f-8eb0-4f150c21cc21 started at 2022-11-24 05:02:13 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container agnhost-container ready: true, restart count 4 Nov 24 05:05:36.992: INFO: rs-npc7z started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:36.992: INFO: inline-volume-tester-vw8hc started at 2022-11-24 05:01:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 24 05:05:36.992: INFO: e2e-dns-utils started at 2022-11-24 05:02:21 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:05:36.992: INFO: csi-mockplugin-0 started at 2022-11-24 05:04:45 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container driver-registrar ready: true, restart count 1 Nov 24 05:05:36.992: INFO: Container mock ready: true, restart count 1 Nov 24 05:05:36.992: INFO: kube-proxy-bootstrap-e2e-minion-group-phcp started at 2022-11-24 04:58:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container kube-proxy ready: true, restart count 4 Nov 24 05:05:36.992: INFO: kube-dns-autoscaler-5f6455f985-94cxr started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container autoscaler ready: false, restart count 4 Nov 24 05:05:36.992: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:36.992: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:36.992: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:36.992: INFO: affinity-clusterip-transition-znjbv started at 2022-11-24 05:02:23 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container affinity-clusterip-transition ready: true, restart count 1 Nov 24 05:05:36.992: INFO: netserver-1 started at 2022-11-24 05:05:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:36.992: INFO: Container webserver ready: false, restart count 1 Nov 24 05:05:37.491: INFO: Latency metrics for node bootstrap-e2e-minion-group-phcp Nov 24 05:05:37.491: INFO: Logging node info for node bootstrap-e2e-minion-group-qvgt Nov 24 05:05:37.551: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 5261 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3073":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-6993":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:03:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:05:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:05:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3073^99fab5e2-6bb5-11ed-a2c8-faefa5088846 kubernetes.io/csi/csi-hostpath-provisioning-9601^22563122-6bb5-11ed-8e10-02ac856346a1],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-9601^22563122-6bb5-11ed-8e10-02ac856346a1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3073^99fab5e2-6bb5-11ed-a2c8-faefa5088846,DevicePath:,},},Config:nil,},} Nov 24 05:05:37.552: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qvgt Nov 24 05:05:37.638: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt Nov 24 05:05:37.762: INFO: fail-once-local-89bj4 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container c ready: false, restart count 1 Nov 24 05:05:37.762: INFO: verify-service-up-host-exec-pod started at 2022-11-24 05:04:55 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container agnhost-container ready: true, restart count 2 Nov 24 05:05:37.762: INFO: kube-proxy-bootstrap-e2e-minion-group-qvgt started at 2022-11-24 04:58:17 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container kube-proxy ready: false, restart count 4 Nov 24 05:05:37.762: INFO: fail-once-local-nf9km started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container c ready: false, restart count 1 Nov 24 05:05:37.762: INFO: rs-xgwnf started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container donothing ready: true, restart count 0 Nov 24 05:05:37.762: INFO: test-pod-3 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container token-test ready: true, restart count 0 Nov 24 05:05:37.762: INFO: pod-subpath-test-dynamicpv-hn5b started at 2022-11-24 05:04:44 +0000 UTC (2+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Init container init-volume-dynamicpv-hn5b ready: false, restart count 0 Nov 24 05:05:37.762: INFO: Init container test-init-volume-dynamicpv-hn5b ready: false, restart count 0 Nov 24 05:05:37.762: INFO: Container test-container-subpath-dynamicpv-hn5b ready: false, restart count 0 Nov 24 05:05:37.762: INFO: metadata-proxy-v0.1-47gbm started at 2022-11-24 04:58:18 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:37.762: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:37.762: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:37.762: INFO: pod-client started at 2022-11-24 05:01:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container pod-client ready: true, restart count 3 Nov 24 05:05:37.762: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-hj9ww started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container agnhost-container ready: true, restart count 1 Nov 24 05:05:37.762: INFO: netserver-2 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:37.762: INFO: netserver-2 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container webserver ready: false, restart count 4 Nov 24 05:05:37.762: INFO: rs-2nnn5 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:37.762: INFO: coredns-6d97d5ddb-r28nc started at 2022-11-24 04:58:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container coredns ready: true, restart count 5 Nov 24 05:05:37.762: INFO: affinity-clusterip-transition-29sd6 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container affinity-clusterip-transition ready: true, restart count 2 Nov 24 05:05:37.762: INFO: pod-2c83f099-b316-4517-bd28-8612b6af4ef4 started at 2022-11-24 05:04:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container write-pod ready: true, restart count 0 Nov 24 05:05:37.762: INFO: external-provisioner-z5v6t started at 2022-11-24 05:01:54 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container nfs-provisioner ready: true, restart count 4 Nov 24 05:05:37.762: INFO: pod-subpath-test-dynamicpv-jv9j started at 2022-11-24 05:02:03 +0000 UTC (2+2 container statuses recorded) Nov 24 05:05:37.762: INFO: Init container init-volume-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:37.762: INFO: Init container test-init-subpath-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:37.762: INFO: Container test-container-subpath-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:37.762: INFO: Container test-container-volume-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:37.762: INFO: test-pod-2 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container token-test ready: false, restart count 0 Nov 24 05:05:37.762: INFO: rs-7mxrd started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:37.762: INFO: service-deletion-jws9m started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container service-deletion ready: true, restart count 2 Nov 24 05:05:37.762: INFO: downwardapi-volume-899fb232-b5a9-4752-a4bf-e80f2a5924e1 started at 2022-11-24 05:02:33 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container client-container ready: false, restart count 0 Nov 24 05:05:37.762: INFO: pvc-volume-tester-writer-dhhpl started at 2022-11-24 05:04:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container volume-tester ready: false, restart count 0 Nov 24 05:05:37.762: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:01:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container csi-attacher ready: true, restart count 1 Nov 24 05:05:37.762: INFO: netserver-2 started at 2022-11-24 05:05:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container webserver ready: true, restart count 1 Nov 24 05:05:37.762: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:57 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:37.762: INFO: Container csi-attacher ready: false, restart count 4 Nov 24 05:05:37.762: INFO: Container csi-provisioner ready: false, restart count 4 Nov 24 05:05:37.762: INFO: Container csi-resizer ready: false, restart count 4 Nov 24 05:05:37.762: INFO: Container csi-snapshotter ready: false, restart count 4 Nov 24 05:05:37.762: INFO: Container hostpath ready: false, restart count 4 Nov 24 05:05:37.762: INFO: Container liveness-probe ready: false, restart count 4 Nov 24 05:05:37.762: INFO: Container node-driver-registrar ready: true, restart count 5 Nov 24 05:05:37.762: INFO: netserver-2 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:37.762: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:05:19 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:37.762: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:37.762: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:37.762: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:37.762: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:37.762: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:37.762: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:37.762: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:37.762: INFO: konnectivity-agent-s27mp started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container konnectivity-agent ready: true, restart count 4 Nov 24 05:05:37.762: INFO: external-provisioner-8vddn started at 2022-11-24 05:02:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container nfs-provisioner ready: false, restart count 4 Nov 24 05:05:37.762: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:37.762: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:05:37.762: INFO: Container driver-registrar ready: true, restart count 1 Nov 24 05:05:37.762: INFO: Container mock ready: true, restart count 1 Nov 24 05:05:37.762: INFO: inline-volume-tester-s8gb9 started at 2022-11-24 05:05:24 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:37.762: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 24 05:05:38.065: INFO: Latency metrics for node bootstrap-e2e-minion-group-qvgt STEP: Collecting events from namespace "webhook-4642-markers". 11/24/22 05:05:38.065 STEP: Found 0 events. 11/24/22 05:05:38.115 Nov 24 05:05:38.198: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 05:05:38.198: INFO: Nov 24 05:05:38.254: INFO: Logging node info for node bootstrap-e2e-master Nov 24 05:05:38.323: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 3828 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:04:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:04:19 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:05:38.324: INFO: Logging kubelet events for node bootstrap-e2e-master Nov 24 05:05:38.390: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Nov 24 05:05:38.469: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container etcd-container ready: true, restart count 2 Nov 24 05:05:38.469: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container kube-apiserver ready: true, restart count 1 Nov 24 05:05:38.469: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-24 04:57:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container kube-addon-manager ready: true, restart count 1 Nov 24 05:05:38.469: INFO: metadata-proxy-v0.1-ww2rj started at 2022-11-24 04:58:45 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:38.469: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:38.469: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:38.469: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-24 04:57:52 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container l7-lb-controller ready: true, restart count 4 Nov 24 05:05:38.469: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container kube-controller-manager ready: true, restart count 4 Nov 24 05:05:38.469: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container kube-scheduler ready: true, restart count 2 Nov 24 05:05:38.469: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container etcd-container ready: true, restart count 1 Nov 24 05:05:38.469: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.469: INFO: Container konnectivity-server-container ready: true, restart count 1 Nov 24 05:05:38.680: INFO: Latency metrics for node bootstrap-e2e-master Nov 24 05:05:38.680: INFO: Logging node info for node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:38.751: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 5082 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1793":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:03:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:04:57 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-24 05:05:21 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:03:24 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:05:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:05:38.752: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:38.818: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:38.966: INFO: rs-fpj4b started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container donothing ready: false, restart count 2 Nov 24 05:05:38.966: INFO: netserver-0 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:38.966: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:04:45 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:38.966: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:12 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:38.966: INFO: Container csi-attacher ready: false, restart count 4 Nov 24 05:05:38.966: INFO: Container csi-provisioner ready: false, restart count 4 Nov 24 05:05:38.966: INFO: Container csi-resizer ready: false, restart count 4 Nov 24 05:05:38.966: INFO: Container csi-snapshotter ready: false, restart count 4 Nov 24 05:05:38.966: INFO: Container hostpath ready: false, restart count 4 Nov 24 05:05:38.966: INFO: Container liveness-probe ready: false, restart count 4 Nov 24 05:05:38.966: INFO: Container node-driver-registrar ready: false, restart count 4 Nov 24 05:05:38.966: INFO: affinity-clusterip-transition-gqrb4 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container affinity-clusterip-transition ready: true, restart count 2 Nov 24 05:05:38.966: INFO: pod-handle-http-request started at 2022-11-24 05:02:33 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:38.966: INFO: Container container-handle-http-request ready: true, restart count 2 Nov 24 05:05:38.966: INFO: Container container-handle-https-request ready: true, restart count 2 Nov 24 05:05:38.966: INFO: external-provisioner-tb7tr started at 2022-11-24 05:05:11 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container nfs-provisioner ready: false, restart count 1 Nov 24 05:05:38.966: INFO: kube-proxy-bootstrap-e2e-minion-group-gz1x started at 2022-11-24 04:58:18 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container kube-proxy ready: true, restart count 4 Nov 24 05:05:38.966: INFO: metrics-server-v0.5.2-867b8754b9-6fxbz started at 2022-11-24 04:59:43 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:38.966: INFO: Container metrics-server ready: false, restart count 3 Nov 24 05:05:38.966: INFO: Container metrics-server-nanny ready: false, restart count 4 Nov 24 05:05:38.966: INFO: rs-k8v86 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:38.966: INFO: netserver-0 started at 2022-11-24 05:05:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container webserver ready: false, restart count 1 Nov 24 05:05:38.966: INFO: test-pod-1 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container token-test ready: false, restart count 0 Nov 24 05:05:38.966: INFO: netserver-0 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container webserver ready: true, restart count 0 Nov 24 05:05:38.966: INFO: csi-mockplugin-0 started at 2022-11-24 05:04:45 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:38.966: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:38.966: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 05:05:38.966: INFO: Container mock ready: true, restart count 0 Nov 24 05:05:38.966: INFO: verify-service-up-exec-pod-76q9m started at 2022-11-24 05:05:07 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:05:38.966: INFO: fail-once-local-hs44m started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container c ready: false, restart count 1 Nov 24 05:05:38.966: INFO: rs-ndmwk started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:38.966: INFO: hostpath-client started at 2022-11-24 05:05:10 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container hostpath-client ready: false, restart count 1 Nov 24 05:05:38.966: INFO: netserver-0 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container webserver ready: true, restart count 0 Nov 24 05:05:38.966: INFO: liveness-d2d53f99-b5a7-4102-9424-8e2aae6c8329 started at 2022-11-24 05:04:55 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.966: INFO: Container agnhost-container ready: true, restart count 2 Nov 24 05:05:38.967: INFO: fail-once-local-2b8gh started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container c ready: false, restart count 1 Nov 24 05:05:38.967: INFO: logs-generator started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container logs-generator ready: false, restart count 0 Nov 24 05:05:38.967: INFO: pod-ready started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container pod-readiness-gate ready: true, restart count 0 Nov 24 05:05:38.967: INFO: pod-tk9n9 started at 2022-11-24 05:02:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container agnhost ready: true, restart count 1 Nov 24 05:05:38.967: INFO: rs-9czwp started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container donothing ready: true, restart count 0 Nov 24 05:05:38.967: INFO: test-webserver-551993bf-4ba0-4a93-92a1-0771237b1b1e started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container test-webserver ready: false, restart count 0 Nov 24 05:05:38.967: INFO: pod-projected-secrets-15b2970c-e6c5-4f3c-bb1d-fe7abc7a3522 started at 2022-11-24 05:05:36 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container projected-secret-volume-test ready: false, restart count 0 Nov 24 05:05:38.967: INFO: test-container-pod started at 2022-11-24 05:05:38 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container webserver ready: false, restart count 0 Nov 24 05:05:38.967: INFO: metadata-proxy-v0.1-f4l4j started at 2022-11-24 04:58:19 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:38.967: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:38.967: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:38.967: INFO: hostexec-bootstrap-e2e-minion-group-gz1x-jcbgl started at 2022-11-24 05:02:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:05:38.967: INFO: test-grpc-1c6bd144-5b65-4cb4-91dc-a99a6f2d8d15 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container etcd ready: true, restart count 2 Nov 24 05:05:38.967: INFO: konnectivity-agent-4tkcz started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container konnectivity-agent ready: true, restart count 4 Nov 24 05:05:38.967: INFO: liveness-ecce2bb9-da3e-402f-8446-00acbacd1ce3 started at 2022-11-24 05:01:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:38.967: INFO: Container agnhost-container ready: true, restart count 3 Nov 24 05:05:39.374: INFO: Latency metrics for node bootstrap-e2e-minion-group-gz1x Nov 24 05:05:39.374: INFO: Logging node info for node bootstrap-e2e-minion-group-phcp Nov 24 05:05:39.450: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 5036 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-3606":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-3727":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-24 05:03:27 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:05:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:05:20 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:03:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:05:20 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},},Config:nil,},} Nov 24 05:05:39.451: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-phcp Nov 24 05:05:39.515: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp Nov 24 05:05:39.675: INFO: kube-proxy-bootstrap-e2e-minion-group-phcp started at 2022-11-24 04:58:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container kube-proxy ready: true, restart count 4 Nov 24 05:05:39.675: INFO: kube-dns-autoscaler-5f6455f985-94cxr started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container autoscaler ready: false, restart count 4 Nov 24 05:05:39.675: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:39.675: INFO: affinity-clusterip-transition-znjbv started at 2022-11-24 05:02:23 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container affinity-clusterip-transition ready: true, restart count 1 Nov 24 05:05:39.675: INFO: netserver-1 started at 2022-11-24 05:05:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container webserver ready: false, restart count 1 Nov 24 05:05:39.675: INFO: coredns-6d97d5ddb-s4dqd started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container coredns ready: true, restart count 3 Nov 24 05:05:39.675: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:39.675: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:04:45 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-attacher ready: true, restart count 1 Nov 24 05:05:39.675: INFO: csi-mockplugin-0 started at 2022-11-24 05:02:32 +0000 UTC (0+4 container statuses recorded) Nov 24 05:05:39.675: INFO: Container busybox ready: true, restart count 3 Nov 24 05:05:39.675: INFO: Container csi-provisioner ready: false, restart count 3 Nov 24 05:05:39.675: INFO: Container driver-registrar ready: false, restart count 3 Nov 24 05:05:39.675: INFO: Container mock ready: false, restart count 3 Nov 24 05:05:39.675: INFO: rs-ppkb5 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container donothing ready: true, restart count 0 Nov 24 05:05:39.675: INFO: rs-6mxtz started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container donothing ready: false, restart count 2 Nov 24 05:05:39.675: INFO: netserver-1 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:39.675: INFO: konnectivity-agent-2t55k started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container konnectivity-agent ready: true, restart count 2 Nov 24 05:05:39.675: INFO: pod-handle-http-request started at 2022-11-24 05:05:03 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:39.675: INFO: Container container-handle-http-request ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container container-handle-https-request ready: true, restart count 0 Nov 24 05:05:39.675: INFO: pod-with-prestop-https-hook started at 2022-11-24 05:05:09 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container pod-with-prestop-https-hook ready: false, restart count 2 Nov 24 05:05:39.675: INFO: pod-server-1 started at 2022-11-24 05:02:22 +0000 UTC (1+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Init container init ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container agnhost-container ready: true, restart count 1 Nov 24 05:05:39.675: INFO: netserver-1 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:39.675: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-provisioner ready: true, restart count 3 Nov 24 05:05:39.675: INFO: Container driver-registrar ready: true, restart count 3 Nov 24 05:05:39.675: INFO: Container mock ready: true, restart count 3 Nov 24 05:05:39.675: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:04:49 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-attacher ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container csi-resizer ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container csi-snapshotter ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container hostpath ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container liveness-probe ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container node-driver-registrar ready: true, restart count 1 Nov 24 05:05:39.675: INFO: metadata-proxy-v0.1-lghrl started at 2022-11-24 04:58:23 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:39.675: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:39.675: INFO: l7-default-backend-8549d69d99-29vxh started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container default-http-backend ready: true, restart count 0 Nov 24 05:05:39.675: INFO: volume-snapshot-controller-0 started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container volume-snapshot-controller ready: true, restart count 2 Nov 24 05:05:39.675: INFO: netserver-1 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container webserver ready: false, restart count 2 Nov 24 05:05:39.675: INFO: inline-volume-tester-htgjr started at 2022-11-24 05:02:36 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:05:39.675: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:30 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:39.675: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:39.675: INFO: e2e-configmap-dns-server-30586570-454b-434f-8eb0-4f150c21cc21 started at 2022-11-24 05:02:13 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container agnhost-container ready: true, restart count 4 Nov 24 05:05:39.675: INFO: rs-npc7z started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:39.675: INFO: inline-volume-tester-vw8hc started at 2022-11-24 05:01:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 24 05:05:39.675: INFO: e2e-dns-utils started at 2022-11-24 05:02:21 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:39.675: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:05:39.675: INFO: csi-mockplugin-0 started at 2022-11-24 05:04:45 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:39.675: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container driver-registrar ready: true, restart count 1 Nov 24 05:05:39.675: INFO: Container mock ready: true, restart count 1 Nov 24 05:05:40.192: INFO: Latency metrics for node bootstrap-e2e-minion-group-phcp Nov 24 05:05:40.192: INFO: Logging node info for node bootstrap-e2e-minion-group-qvgt Nov 24 05:05:40.280: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 5261 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3073":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-6993":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:03:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:05:24 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:05:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KernelHasNoDeadlock,Message:kernel has no deadlock,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:03:21 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:05:31 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-3073^99fab5e2-6bb5-11ed-a2c8-faefa5088846 kubernetes.io/csi/csi-hostpath-provisioning-9601^22563122-6bb5-11ed-8e10-02ac856346a1],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-9601^22563122-6bb5-11ed-8e10-02ac856346a1,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-3073^99fab5e2-6bb5-11ed-a2c8-faefa5088846,DevicePath:,},},Config:nil,},} Nov 24 05:05:40.281: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qvgt Nov 24 05:05:40.361: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt Nov 24 05:05:40.487: INFO: kube-proxy-bootstrap-e2e-minion-group-qvgt started at 2022-11-24 04:58:17 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container kube-proxy ready: false, restart count 4 Nov 24 05:05:40.487: INFO: fail-once-local-nf9km started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container c ready: false, restart count 1 Nov 24 05:05:40.487: INFO: metadata-proxy-v0.1-47gbm started at 2022-11-24 04:58:18 +0000 UTC (0+2 container statuses recorded) Nov 24 05:05:40.487: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:05:40.487: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:05:40.487: INFO: pod-client started at 2022-11-24 05:01:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container pod-client ready: true, restart count 3 Nov 24 05:05:40.487: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-hj9ww started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container agnhost-container ready: true, restart count 1 Nov 24 05:05:40.487: INFO: rs-xgwnf started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container donothing ready: true, restart count 0 Nov 24 05:05:40.487: INFO: test-pod-3 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container token-test ready: true, restart count 0 Nov 24 05:05:40.487: INFO: pod-subpath-test-dynamicpv-hn5b started at 2022-11-24 05:04:44 +0000 UTC (2+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Init container init-volume-dynamicpv-hn5b ready: false, restart count 0 Nov 24 05:05:40.487: INFO: Init container test-init-volume-dynamicpv-hn5b ready: false, restart count 0 Nov 24 05:05:40.487: INFO: Container test-container-subpath-dynamicpv-hn5b ready: false, restart count 0 Nov 24 05:05:40.487: INFO: netserver-2 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container webserver ready: false, restart count 4 Nov 24 05:05:40.487: INFO: rs-2nnn5 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:40.487: INFO: netserver-2 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:40.487: INFO: coredns-6d97d5ddb-r28nc started at 2022-11-24 04:58:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container coredns ready: true, restart count 5 Nov 24 05:05:40.487: INFO: affinity-clusterip-transition-29sd6 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container affinity-clusterip-transition ready: true, restart count 2 Nov 24 05:05:40.487: INFO: pod-2c83f099-b316-4517-bd28-8612b6af4ef4 started at 2022-11-24 05:04:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container write-pod ready: true, restart count 0 Nov 24 05:05:40.487: INFO: downwardapi-volume-899fb232-b5a9-4752-a4bf-e80f2a5924e1 started at 2022-11-24 05:02:33 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container client-container ready: false, restart count 0 Nov 24 05:05:40.487: INFO: pvc-volume-tester-writer-dhhpl started at 2022-11-24 05:04:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container volume-tester ready: false, restart count 0 Nov 24 05:05:40.487: INFO: external-provisioner-z5v6t started at 2022-11-24 05:01:54 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container nfs-provisioner ready: true, restart count 4 Nov 24 05:05:40.487: INFO: pod-subpath-test-dynamicpv-jv9j started at 2022-11-24 05:02:03 +0000 UTC (2+2 container statuses recorded) Nov 24 05:05:40.487: INFO: Init container init-volume-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:40.487: INFO: Init container test-init-subpath-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:40.487: INFO: Container test-container-subpath-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:40.487: INFO: Container test-container-volume-dynamicpv-jv9j ready: false, restart count 0 Nov 24 05:05:40.487: INFO: test-pod-2 started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container token-test ready: false, restart count 0 Nov 24 05:05:40.487: INFO: rs-7mxrd started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container donothing ready: true, restart count 2 Nov 24 05:05:40.487: INFO: service-deletion-jws9m started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container service-deletion ready: true, restart count 2 Nov 24 05:05:40.487: INFO: konnectivity-agent-s27mp started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container konnectivity-agent ready: true, restart count 4 Nov 24 05:05:40.487: INFO: external-provisioner-8vddn started at 2022-11-24 05:02:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container nfs-provisioner ready: false, restart count 4 Nov 24 05:05:40.487: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+3 container statuses recorded) Nov 24 05:05:40.487: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:05:40.487: INFO: Container driver-registrar ready: true, restart count 1 Nov 24 05:05:40.487: INFO: Container mock ready: true, restart count 1 Nov 24 05:05:40.487: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:01:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container csi-attacher ready: true, restart count 1 Nov 24 05:05:40.487: INFO: netserver-2 started at 2022-11-24 05:05:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container webserver ready: true, restart count 1 Nov 24 05:05:40.487: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:57 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:40.487: INFO: Container csi-attacher ready: false, restart count 4 Nov 24 05:05:40.487: INFO: Container csi-provisioner ready: false, restart count 4 Nov 24 05:05:40.487: INFO: Container csi-resizer ready: false, restart count 4 Nov 24 05:05:40.487: INFO: Container csi-snapshotter ready: false, restart count 4 Nov 24 05:05:40.487: INFO: Container hostpath ready: false, restart count 4 Nov 24 05:05:40.487: INFO: Container liveness-probe ready: false, restart count 4 Nov 24 05:05:40.487: INFO: Container node-driver-registrar ready: true, restart count 5 Nov 24 05:05:40.487: INFO: netserver-2 started at 2022-11-24 05:04:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container webserver ready: true, restart count 2 Nov 24 05:05:40.487: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:05:19 +0000 UTC (0+7 container statuses recorded) Nov 24 05:05:40.487: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:05:40.487: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:05:40.487: INFO: Container csi-resizer ready: true, restart count 0 Nov 24 05:05:40.487: INFO: Container csi-snapshotter ready: true, restart count 0 Nov 24 05:05:40.487: INFO: Container hostpath ready: true, restart count 0 Nov 24 05:05:40.487: INFO: Container liveness-probe ready: true, restart count 0 Nov 24 05:05:40.487: INFO: Container node-driver-registrar ready: true, restart count 0 Nov 24 05:05:40.487: INFO: inline-volume-tester-s8gb9 started at 2022-11-24 05:05:24 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container csi-volume-tester ready: true, restart count 0 Nov 24 05:05:40.487: INFO: fail-once-local-89bj4 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container c ready: false, restart count 1 Nov 24 05:05:40.487: INFO: verify-service-up-host-exec-pod started at 2022-11-24 05:04:55 +0000 UTC (0+1 container statuses recorded) Nov 24 05:05:40.487: INFO: Container agnhost-container ready: true, restart count 2 Nov 24 05:05:40.932: INFO: Latency metrics for node bootstrap-e2e-minion-group-qvgt [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] tear down framework | framework.go:193 STEP: Destroying namespace "webhook-4642" for this suite. 11/24/22 05:05:40.933 STEP: Destroying namespace "webhook-4642-markers" for this suite. 11/24/22 05:05:41.073
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sbe\sable\sto\sdeny\sattaching\spod\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0012abef0)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:31:21.792
Nov 24 05:31:21.792: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:31:21.793
Nov 24 05:31:21.832: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:23.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:25.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:27.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:29.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:31.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:33.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:35.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:37.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:39.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:41.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:43.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:45.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:47.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:49.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:51.872: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:51.911: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:51.911: INFO: Unexpected error:
<*errors.errorString | 0xc0002559e0>: {
s: "timed out waiting for the condition",
}
Nov 24 05:31:51.911: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0012abef0)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:31:51.911: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:31:51.951
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sbe\sable\sto\sdeny\scustom\sresource\screation\,\supdate\sand\sdeletion\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0007a3e00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:32:47.681
Nov 24 05:32:47.682: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:32:47.683
Nov 24 05:32:47.723: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:49.762: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:51.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:53.762: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:55.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:57.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:59.762: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:01.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:03.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:05.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:07.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:09.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:11.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:13.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:15.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:17.763: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:17.802: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:17.802: INFO: Unexpected error:
<*errors.errorString | 0xc000205d10>: {
s: "timed out waiting for the condition",
}
Nov 24 05:33:17.802: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0007a3e00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:33:17.803: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:33:17.842
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sbe\sable\sto\sdeny\spod\sand\sconfigmap\screation\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000e4de00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:33:52.716
Nov 24 05:33:52.716: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:33:52.718
Nov 24 05:33:52.757: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:54.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:56.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:58.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:00.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:02.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:04.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:06.796: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:08.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:10.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:12.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:14.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:16.798: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:18.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:20.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:22.797: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:22.836: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:34:22.836: INFO: Unexpected error:
<*errors.errorString | 0xc000205cb0>: {
s: "timed out waiting for the condition",
}
Nov 24 05:34:22.836: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000e4de00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:34:22.836: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:34:22.876
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sdeny\scrd\screation\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001195e00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:31:53.455
Nov 24 05:31:53.455: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:31:53.457
Nov 24 05:31:53.496: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:55.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:57.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:59.537: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:01.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:03.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:05.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:07.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:09.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:11.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:13.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:15.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:17.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:19.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:21.536: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:23.535: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:23.575: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:23.575: INFO: Unexpected error:
<*errors.errorString | 0xc000111cd0>: {
s: "timed out waiting for the condition",
}
Nov 24 05:32:23.575: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001195e00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:32:23.575: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:32:23.615
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\shonor\stimeout\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0003a3950) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:35:21.769 Nov 24 05:35:21.769: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename webhook 11/24/22 05:35:21.771 Nov 24 05:35:21.811: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:23.851: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:37:30.378: INFO: Unexpected error: <*fmt.wrapError | 0xc001632020>: { msg: "wait for service account \"default\" in namespace \"webhook-1880\": timed out waiting for the condition", err: <*errors.errorString | 0xc0000d1db0>{ s: "timed out waiting for the condition", }, } Nov 24 05:37:30.378: FAIL: wait for service account "default" in namespace "webhook-1880": timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0003a3950) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:37:30.379: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] test/e2e/apimachinery/webhook.go:105 [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:37:31.027 STEP: Collecting events from namespace "webhook-1880". 11/24/22 05:37:31.027 STEP: Found 0 events. 11/24/22 05:37:31.075 Nov 24 05:37:31.154: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 05:37:31.154: INFO: Nov 24 05:37:31.203: INFO: Logging node info for node bootstrap-e2e-master Nov 24 05:37:31.244: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 17584 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:35:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:37:31.245: INFO: Logging kubelet events for node bootstrap-e2e-master Nov 24 05:37:31.398: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Nov 24 05:37:31.722: INFO: metadata-proxy-v0.1-ww2rj started at 2022-11-24 04:58:45 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:31.722: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:31.722: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:31.722: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container etcd-container ready: true, restart count 3 Nov 24 05:37:31.722: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container kube-apiserver ready: true, restart count 5 Nov 24 05:37:31.722: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-24 04:57:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container kube-addon-manager ready: true, restart count 5 Nov 24 05:37:31.722: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container konnectivity-server-container ready: true, restart count 6 Nov 24 05:37:31.722: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-24 04:57:52 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container l7-lb-controller ready: false, restart count 9 Nov 24 05:37:31.722: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container kube-controller-manager ready: false, restart count 9 Nov 24 05:37:31.722: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container kube-scheduler ready: true, restart count 7 Nov 24 05:37:31.722: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:31.722: INFO: Container etcd-container ready: true, restart count 3 Nov 24 05:37:32.800: INFO: Latency metrics for node bootstrap-e2e-master Nov 24 05:37:32.800: INFO: Logging node info for node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:32.860: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 17889 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-ephemeral-9642":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-volume-expand-3803":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-3519":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8250":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8355":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:37:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:37:32.860: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:32.947: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:33.360: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:12 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-attacher ready: true, restart count 11 Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: true, restart count 11 Nov 24 05:37:33.360: INFO: Container csi-resizer ready: true, restart count 11 Nov 24 05:37:33.360: INFO: Container csi-snapshotter ready: true, restart count 11 Nov 24 05:37:33.360: INFO: Container hostpath ready: true, restart count 11 Nov 24 05:37:33.360: INFO: Container liveness-probe ready: true, restart count 11 Nov 24 05:37:33.360: INFO: Container node-driver-registrar ready: true, restart count 10 Nov 24 05:37:33.360: INFO: affinity-clusterip-transition-gqrb4 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container affinity-clusterip-transition ready: true, restart count 8 Nov 24 05:37:33.360: INFO: inline-volume-tester-dsr79 started at 2022-11-24 05:27:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:33.360: INFO: pod-subpath-test-projected-68ws started at 2022-11-24 05:29:23 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container test-container-subpath-projected-68ws ready: false, restart count 0 Nov 24 05:37:33.360: INFO: ss2-0 started at 2022-11-24 05:29:18 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 0 Nov 24 05:37:33.360: INFO: sysctl-99b91704-a59b-41b5-9b20-83e75bec93f2 started at 2022-11-24 05:13:47 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container test-container ready: false, restart count 0 Nov 24 05:37:33.360: INFO: test-container-pod started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:33.360: INFO: pod-1 started at 2022-11-24 05:29:08 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 05:37:33.360: INFO: netserver-0 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:33.360: INFO: hostexec-bootstrap-e2e-minion-group-gz1x-jcbgl started at 2022-11-24 05:02:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container agnhost-container ready: false, restart count 7 Nov 24 05:37:33.360: INFO: host-test-container-pod started at 2022-11-24 05:22:59 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container agnhost-container ready: true, restart count 4 Nov 24 05:37:33.360: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-attacher ready: true, restart count 2 Nov 24 05:37:33.360: INFO: test-container-pod started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:33.360: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:29:33 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:37:33.360: INFO: test-rs-nd7pk started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container httpd ready: true, restart count 2 Nov 24 05:37:33.360: INFO: netserver-0 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: false, restart count 5 Nov 24 05:37:33.360: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:13:34 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-attacher ready: true, restart count 8 Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: true, restart count 8 Nov 24 05:37:33.360: INFO: Container csi-resizer ready: true, restart count 8 Nov 24 05:37:33.360: INFO: Container csi-snapshotter ready: true, restart count 8 Nov 24 05:37:33.360: INFO: Container hostpath ready: true, restart count 8 Nov 24 05:37:33.360: INFO: Container liveness-probe ready: true, restart count 8 Nov 24 05:37:33.360: INFO: Container node-driver-registrar ready: true, restart count 8 Nov 24 05:37:33.360: INFO: test-ss-0 started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 5 Nov 24 05:37:33.360: INFO: csi-mockplugin-0 started at 2022-11-24 05:29:33 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: true, restart count 2 Nov 24 05:37:33.360: INFO: Container driver-registrar ready: true, restart count 2 Nov 24 05:37:33.360: INFO: Container mock ready: true, restart count 2 Nov 24 05:37:33.360: INFO: slow-terminating-unready-pod-k5dsj started at 2022-11-24 05:29:03 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container slow-terminating-unready-pod ready: false, restart count 0 Nov 24 05:37:33.360: INFO: netserver-0 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: false, restart count 9 Nov 24 05:37:33.360: INFO: kube-proxy-bootstrap-e2e-minion-group-gz1x started at 2022-11-24 04:58:18 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container kube-proxy ready: false, restart count 9 Nov 24 05:37:33.360: INFO: metrics-server-v0.5.2-867b8754b9-6fxbz started at 2022-11-24 04:59:43 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:33.360: INFO: Container metrics-server ready: false, restart count 9 Nov 24 05:37:33.360: INFO: Container metrics-server-nanny ready: false, restart count 10 Nov 24 05:37:33.360: INFO: pod-handle-http-request started at 2022-11-24 05:02:33 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:33.360: INFO: Container container-handle-http-request ready: true, restart count 9 Nov 24 05:37:33.360: INFO: Container container-handle-https-request ready: true, restart count 9 Nov 24 05:37:33.360: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:01 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:37:33.360: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 05:37:33.360: INFO: Container mock ready: true, restart count 0 Nov 24 05:37:33.360: INFO: external-provisioner-rs6wm started at 2022-11-24 05:29:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container nfs-provisioner ready: false, restart count 5 Nov 24 05:37:33.360: INFO: test-container-pod started at 2022-11-24 05:28:40 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 0 Nov 24 05:37:33.360: INFO: inline-volume-tester-zdzgd started at 2022-11-24 05:29:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:33.360: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:54 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: false, restart count 8 Nov 24 05:37:33.360: INFO: Container driver-registrar ready: false, restart count 8 Nov 24 05:37:33.360: INFO: Container mock ready: false, restart count 8 Nov 24 05:37:33.360: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:54 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-attacher ready: true, restart count 4 Nov 24 05:37:33.360: INFO: fail-once-local-hs44m started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container c ready: false, restart count 1 Nov 24 05:37:33.360: INFO: netserver-0 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: false, restart count 6 Nov 24 05:37:33.360: INFO: fail-once-local-2b8gh started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container c ready: false, restart count 1 Nov 24 05:37:33.360: INFO: csi-mockplugin-0 started at 2022-11-24 05:15:00 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: true, restart count 9 Nov 24 05:37:33.360: INFO: Container driver-registrar ready: true, restart count 11 Nov 24 05:37:33.360: INFO: Container mock ready: true, restart count 9 Nov 24 05:37:33.360: INFO: ss2-2 started at 2022-11-24 05:27:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 6 Nov 24 05:37:33.360: INFO: external-provisioner-dhzp9 started at 2022-11-24 05:27:36 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container nfs-provisioner ready: false, restart count 5 Nov 24 05:37:33.360: INFO: netserver-0 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:33.360: INFO: metadata-proxy-v0.1-f4l4j started at 2022-11-24 04:58:19 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:33.360: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:33.360: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:33.360: INFO: netserver-0 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:33.360: INFO: pod-tk9n9 started at 2022-11-24 05:02:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container agnhost ready: false, restart count 7 Nov 24 05:37:33.360: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:27:27 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-attacher ready: true, restart count 5 Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: true, restart count 5 Nov 24 05:37:33.360: INFO: Container csi-resizer ready: true, restart count 5 Nov 24 05:37:33.360: INFO: Container csi-snapshotter ready: true, restart count 5 Nov 24 05:37:33.360: INFO: Container hostpath ready: true, restart count 5 Nov 24 05:37:33.360: INFO: Container liveness-probe ready: true, restart count 5 Nov 24 05:37:33.360: INFO: Container node-driver-registrar ready: true, restart count 5 Nov 24 05:37:33.360: INFO: httpd started at 2022-11-24 05:29:24 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container httpd ready: false, restart count 2 Nov 24 05:37:33.360: INFO: konnectivity-agent-4tkcz started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container konnectivity-agent ready: false, restart count 9 Nov 24 05:37:33.360: INFO: liveness-ecce2bb9-da3e-402f-8446-00acbacd1ce3 started at 2022-11-24 05:01:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container agnhost-container ready: false, restart count 6 Nov 24 05:37:33.360: INFO: netserver-0 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container webserver ready: true, restart count 2 Nov 24 05:37:33.360: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:51 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:33.360: INFO: Container csi-provisioner ready: true, restart count 5 Nov 24 05:37:33.360: INFO: Container driver-registrar ready: true, restart count 6 Nov 24 05:37:33.360: INFO: Container mock ready: true, restart count 5 Nov 24 05:37:33.360: INFO: replace-27821129-fb7zz started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:33.360: INFO: Container c ready: false, restart count 0 Nov 24 05:37:36.580: INFO: Latency metrics for node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:36.580: INFO: Logging node info for node bootstrap-e2e-minion-group-phcp Nov 24 05:37:36.621: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 17725 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-4441":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-4867":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-8063":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-24 05:35:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:36:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},} Nov 24 05:37:36.622: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-phcp Nov 24 05:37:36.667: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp Nov 24 05:37:36.915: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: true, restart count 6 Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: true, restart count 6 Nov 24 05:37:36.915: INFO: Container csi-resizer ready: true, restart count 6 Nov 24 05:37:36.915: INFO: Container csi-snapshotter ready: true, restart count 6 Nov 24 05:37:36.915: INFO: Container hostpath ready: true, restart count 6 Nov 24 05:37:36.915: INFO: Container liveness-probe ready: true, restart count 6 Nov 24 05:37:36.915: INFO: Container node-driver-registrar ready: true, restart count 6 Nov 24 05:37:36.915: INFO: csi-mockplugin-0 started at 2022-11-24 05:28:07 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:37:36.915: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 05:37:36.915: INFO: Container mock ready: true, restart count 0 Nov 24 05:37:36.915: INFO: test-rs-8tt4l started at 2022-11-24 05:28:49 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container httpd ready: true, restart count 2 Nov 24 05:37:36.915: INFO: netserver-1 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: true, restart count 9 Nov 24 05:37:36.915: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: false, restart count 8 Nov 24 05:37:36.915: INFO: Container driver-registrar ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container mock ready: false, restart count 8 Nov 24 05:37:36.915: INFO: nfs-server started at 2022-11-24 05:29:13 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container nfs-server ready: true, restart count 3 Nov 24 05:37:36.915: INFO: inline-volume-tester-htgjr started at 2022-11-24 05:02:36 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:36.915: INFO: metadata-proxy-v0.1-lghrl started at 2022-11-24 04:58:23 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:36.915: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:36.915: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:36.915: INFO: l7-default-backend-8549d69d99-29vxh started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container default-http-backend ready: true, restart count 0 Nov 24 05:37:36.915: INFO: volume-snapshot-controller-0 started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container volume-snapshot-controller ready: false, restart count 11 Nov 24 05:37:36.915: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:52 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:37:36.915: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:30 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container csi-resizer ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container csi-snapshotter ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container hostpath ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container liveness-probe ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container node-driver-registrar ready: false, restart count 9 Nov 24 05:37:36.915: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:08 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:36.915: INFO: Container csi-resizer ready: false, restart count 6 Nov 24 05:37:36.915: INFO: Container csi-snapshotter ready: false, restart count 6 Nov 24 05:37:36.915: INFO: Container hostpath ready: false, restart count 6 Nov 24 05:37:36.915: INFO: Container liveness-probe ready: false, restart count 6 Nov 24 05:37:36.915: INFO: Container node-driver-registrar ready: false, restart count 6 Nov 24 05:37:36.915: INFO: netserver-1 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: true, restart count 4 Nov 24 05:37:36.915: INFO: inline-volume-tester-7nvw4 started at 2022-11-24 05:27:46 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:36.915: INFO: netserver-1 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: true, restart count 8 Nov 24 05:37:36.915: INFO: ss2-1 started at 2022-11-24 05:29:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:36.915: INFO: nodeport-test-mbqbt started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container nodeport-test ready: false, restart count 2 Nov 24 05:37:36.915: INFO: ss2-1 started at 2022-11-24 05:27:06 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: false, restart count 5 Nov 24 05:37:36.915: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container csi-resizer ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container csi-snapshotter ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container hostpath ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container liveness-probe ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container node-driver-registrar ready: false, restart count 7 Nov 24 05:37:36.915: INFO: affinity-clusterip-transition-znjbv started at 2022-11-24 05:02:23 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container affinity-clusterip-transition ready: false, restart count 8 Nov 24 05:37:36.915: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:33 +0000 UTC (0+4 container statuses recorded) Nov 24 05:37:36.915: INFO: Container busybox ready: false, restart count 5 Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: false, restart count 4 Nov 24 05:37:36.915: INFO: Container driver-registrar ready: false, restart count 5 Nov 24 05:37:36.915: INFO: Container mock ready: false, restart count 5 Nov 24 05:37:36.915: INFO: coredns-6d97d5ddb-s4dqd started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container coredns ready: false, restart count 11 Nov 24 05:37:36.915: INFO: test-container-pod started at 2022-11-24 05:23:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: true, restart count 5 Nov 24 05:37:36.915: INFO: csi-mockplugin-0 started at 2022-11-24 05:02:32 +0000 UTC (0+4 container statuses recorded) Nov 24 05:37:36.915: INFO: Container busybox ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container driver-registrar ready: false, restart count 9 Nov 24 05:37:36.915: INFO: Container mock ready: false, restart count 9 Nov 24 05:37:36.915: INFO: netserver-1 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: true, restart count 6 Nov 24 05:37:36.915: INFO: konnectivity-agent-2t55k started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container konnectivity-agent ready: true, restart count 9 Nov 24 05:37:36.915: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:05:48 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: true, restart count 9 Nov 24 05:37:36.915: INFO: pod-9a7c7833-f26a-4018-9047-bf9628109eaf started at 2022-11-24 05:28:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container write-pod ready: false, restart count 0 Nov 24 05:37:36.915: INFO: hostexec-bootstrap-e2e-minion-group-phcp-6lwvm started at 2022-11-24 05:29:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container agnhost-container ready: false, restart count 2 Nov 24 05:37:36.915: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:52 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:37:36.915: INFO: Container driver-registrar ready: true, restart count 1 Nov 24 05:37:36.915: INFO: Container mock ready: true, restart count 1 Nov 24 05:37:36.915: INFO: pod-server-1 started at 2022-11-24 05:02:22 +0000 UTC (1+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Init container init ready: true, restart count 2 Nov 24 05:37:36.915: INFO: Container agnhost-container ready: true, restart count 10 Nov 24 05:37:36.915: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:35 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container csi-resizer ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container csi-snapshotter ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container hostpath ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container liveness-probe ready: true, restart count 7 Nov 24 05:37:36.915: INFO: Container node-driver-registrar ready: true, restart count 8 Nov 24 05:37:36.915: INFO: netserver-1 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: false, restart count 3 Nov 24 05:37:36.915: INFO: netserver-1 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: false, restart count 5 Nov 24 05:37:36.915: INFO: inline-volume-tester-vw8hc started at 2022-11-24 05:01:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:36.915: INFO: netserver-1 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:36.915: INFO: inline-volume-tester-cbr4h started at 2022-11-24 05:08:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:36.915: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:28:07 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-attacher ready: true, restart count 2 Nov 24 05:37:36.915: INFO: kube-proxy-bootstrap-e2e-minion-group-phcp started at 2022-11-24 04:58:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container kube-proxy ready: false, restart count 9 Nov 24 05:37:36.915: INFO: kube-dns-autoscaler-5f6455f985-94cxr started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:36.915: INFO: Container autoscaler ready: false, restart count 9 Nov 24 05:37:36.915: INFO: csi-mockplugin-0 started at 2022-11-24 05:05:47 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:36.915: INFO: Container csi-provisioner ready: false, restart count 7 Nov 24 05:37:36.915: INFO: Container driver-registrar ready: false, restart count 7 Nov 24 05:37:36.915: INFO: Container mock ready: false, restart count 7 Nov 24 05:37:38.882: INFO: Latency metrics for node bootstrap-e2e-minion-group-phcp Nov 24 05:37:38.882: INFO: Logging node info for node bootstrap-e2e-minion-group-qvgt Nov 24 05:37:38.929: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 17685 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-9601":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:37:38.929: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qvgt Nov 24 05:37:38.974: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt Nov 24 05:37:39.213: INFO: netserver-2 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.213: INFO: Container webserver ready: true, restart count 4 Nov 24 05:37:39.213: INFO: nfs-server started at 2022-11-24 05:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.213: INFO: Container nfs-server ready: true, restart count 0 Nov 24 05:37:39.213: INFO: affinity-clusterip-transition-29sd6 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container affinity-clusterip-transition ready: true, restart count 7 Nov 24 05:37:39.214: INFO: downwardapi-volume-899fb232-b5a9-4752-a4bf-e80f2a5924e1 started at 2022-11-24 05:02:33 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container client-container ready: false, restart count 0 Nov 24 05:37:39.214: INFO: netserver-2 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: false, restart count 6 Nov 24 05:37:39.214: INFO: konnectivity-agent-s27mp started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container konnectivity-agent ready: false, restart count 9 Nov 24 05:37:39.214: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-provisioner ready: false, restart count 9 Nov 24 05:37:39.214: INFO: Container driver-registrar ready: false, restart count 9 Nov 24 05:37:39.214: INFO: Container mock ready: false, restart count 9 Nov 24 05:37:39.214: INFO: ss2-0 started at 2022-11-24 05:27:01 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:39.214: INFO: netserver-2 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: true, restart count 2 Nov 24 05:37:39.214: INFO: netserver-2 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: true, restart count 2 Nov 24 05:37:39.214: INFO: kube-proxy-bootstrap-e2e-minion-group-qvgt started at 2022-11-24 04:58:17 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container kube-proxy ready: false, restart count 10 Nov 24 05:37:39.214: INFO: fail-once-local-nf9km started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container c ready: false, restart count 1 Nov 24 05:37:39.214: INFO: nodeport-test-p2clc started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container nodeport-test ready: true, restart count 4 Nov 24 05:37:39.214: INFO: netserver-2 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:39.214: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-lqgzg started at 2022-11-24 05:29:06 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container agnhost-container ready: true, restart count 1 Nov 24 05:37:39.214: INFO: netserver-2 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: false, restart count 9 Nov 24 05:37:39.214: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:21 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-attacher ready: true, restart count 5 Nov 24 05:37:39.214: INFO: Container csi-provisioner ready: true, restart count 5 Nov 24 05:37:39.214: INFO: Container csi-resizer ready: true, restart count 5 Nov 24 05:37:39.214: INFO: Container csi-snapshotter ready: true, restart count 5 Nov 24 05:37:39.214: INFO: Container hostpath ready: true, restart count 5 Nov 24 05:37:39.214: INFO: Container liveness-probe ready: true, restart count 5 Nov 24 05:37:39.214: INFO: Container node-driver-registrar ready: true, restart count 5 Nov 24 05:37:39.214: INFO: coredns-6d97d5ddb-r28nc started at 2022-11-24 04:58:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container coredns ready: false, restart count 11 Nov 24 05:37:39.214: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:14:23 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container csi-resizer ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container csi-snapshotter ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container hostpath ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container liveness-probe ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container node-driver-registrar ready: false, restart count 6 Nov 24 05:37:39.214: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:39 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container csi-resizer ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container csi-snapshotter ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container hostpath ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container liveness-probe ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container node-driver-registrar ready: false, restart count 6 Nov 24 05:37:39.214: INFO: execpoddrdpm started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:37:39.214: INFO: pvc-volume-tester-writer-dhhpl started at 2022-11-24 05:04:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container volume-tester ready: false, restart count 0 Nov 24 05:37:39.214: INFO: pod-configmaps-c1868fc3-ec52-478d-ad0f-36361860846b started at 2022-11-24 05:26:45 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 05:37:39.214: INFO: test-rs-nr57v started at 2022-11-24 05:28:49 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:39.214: INFO: Container httpd ready: true, restart count 3 Nov 24 05:37:39.214: INFO: Container test-rs ready: true, restart count 3 Nov 24 05:37:39.214: INFO: ss2-2 started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: true, restart count 4 Nov 24 05:37:39.214: INFO: external-provisioner-8vddn started at 2022-11-24 05:02:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container nfs-provisioner ready: true, restart count 8 Nov 24 05:37:39.214: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:01:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-attacher ready: true, restart count 7 Nov 24 05:37:39.214: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:57 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-attacher ready: true, restart count 10 Nov 24 05:37:39.214: INFO: Container csi-provisioner ready: true, restart count 10 Nov 24 05:37:39.214: INFO: Container csi-resizer ready: true, restart count 10 Nov 24 05:37:39.214: INFO: Container csi-snapshotter ready: true, restart count 10 Nov 24 05:37:39.214: INFO: Container hostpath ready: true, restart count 10 Nov 24 05:37:39.214: INFO: Container liveness-probe ready: true, restart count 10 Nov 24 05:37:39.214: INFO: Container node-driver-registrar ready: true, restart count 11 Nov 24 05:37:39.214: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:05:19 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-attacher ready: false, restart count 8 Nov 24 05:37:39.214: INFO: Container csi-provisioner ready: false, restart count 8 Nov 24 05:37:39.214: INFO: Container csi-resizer ready: false, restart count 8 Nov 24 05:37:39.214: INFO: Container csi-snapshotter ready: false, restart count 8 Nov 24 05:37:39.214: INFO: Container hostpath ready: false, restart count 8 Nov 24 05:37:39.214: INFO: Container liveness-probe ready: false, restart count 8 Nov 24 05:37:39.214: INFO: Container node-driver-registrar ready: false, restart count 8 Nov 24 05:37:39.214: INFO: fail-once-local-89bj4 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container c ready: false, restart count 1 Nov 24 05:37:39.214: INFO: netserver-2 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container webserver ready: true, restart count 5 Nov 24 05:37:39.214: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:26 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container driver-registrar ready: false, restart count 6 Nov 24 05:37:39.214: INFO: Container mock ready: false, restart count 6 Nov 24 05:37:39.214: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:26 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:39.214: INFO: metadata-proxy-v0.1-47gbm started at 2022-11-24 04:58:18 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:39.214: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:39.214: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:39.214: INFO: pod-client started at 2022-11-24 05:01:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container pod-client ready: true, restart count 10 Nov 24 05:37:39.214: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-vnr2m started at 2022-11-24 05:28:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.214: INFO: Container agnhost-container ready: false, restart count 5 Nov 24 05:37:39.675: INFO: Latency metrics for node bootstrap-e2e-minion-group-qvgt [DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] tear down framework | framework.go:193 STEP: Destroying namespace "webhook-1880" for this suite. 11/24/22 05:37:39.675
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sinclude\swebhook\sresources\sin\sdiscovery\sdocuments\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001131e00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:37:41.761
Nov 24 05:37:41.761: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:37:41.762
Nov 24 05:39:41.816: INFO: Unexpected error:
<*fmt.wrapError | 0xc000044200>: {
msg: "wait for service account \"default\" in namespace \"webhook-1745\": timed out waiting for the condition",
err: <*errors.errorString | 0xc000215d80>{
s: "timed out waiting for the condition",
},
}
Nov 24 05:39:41.816: FAIL: wait for service account "default" in namespace "webhook-1745": timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001131e00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:39:41.817: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:39:41.862
STEP: Collecting events from namespace "webhook-1745". 11/24/22 05:39:41.862
STEP: Found 0 events. 11/24/22 05:39:41.904
Nov 24 05:39:41.945: INFO: POD NODE PHASE GRACE CONDITIONS
Nov 24 05:39:41.945: INFO:
Nov 24 05:39:41.987: INFO:
Logging node info for node bootstrap-e2e-master
Nov 24 05:39:42.031: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 17584 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:35:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:39:42.031: INFO:
Logging kubelet events for node bootstrap-e2e-master
Nov 24 05:39:42.079: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 24 05:39:42.136: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container etcd-container ready: true, restart count 4
Nov 24 05:39:42.136: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container kube-apiserver ready: true, restart count 5
Nov 24 05:39:42.136: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-24 04:57:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container kube-addon-manager ready: true, restart count 5
Nov 24 05:39:42.136: INFO: metadata-proxy-v0.1-ww2rj started at 2022-11-24 04:58:45 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:39:42.136: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:39:42.136: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container kube-controller-manager ready: false, restart count 9
Nov 24 05:39:42.136: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container kube-scheduler ready: true, restart count 7
Nov 24 05:39:42.136: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container etcd-container ready: true, restart count 3
Nov 24 05:39:42.136: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container konnectivity-server-container ready: true, restart count 6
Nov 24 05:39:42.136: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-24 04:57:52 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:42.136: INFO: Container l7-lb-controller ready: false, restart count 9
Nov 24 05:39:42.463: INFO:
Latency metrics for node bootstrap-e2e-master
Nov 24 05:39:42.463: INFO:
Logging node info for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:39:42.649: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 18167 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-ephemeral-9642":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-volume-expand-3803":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-3519":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8355":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:39:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:39:42.649: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:39:42.890: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x
Nov 24 05:39:43.025: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:51 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: false, restart count 5
Nov 24 05:39:43.025: INFO: Container driver-registrar ready: false, restart count 6
Nov 24 05:39:43.025: INFO: Container mock ready: false, restart count 5
Nov 24 05:39:43.025: INFO: replace-27821129-fb7zz started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container c ready: false, restart count 0
Nov 24 05:39:43.025: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:12 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-attacher ready: true, restart count 11
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: true, restart count 11
Nov 24 05:39:43.025: INFO: Container csi-resizer ready: true, restart count 11
Nov 24 05:39:43.025: INFO: Container csi-snapshotter ready: true, restart count 11
Nov 24 05:39:43.025: INFO: Container hostpath ready: true, restart count 11
Nov 24 05:39:43.025: INFO: Container liveness-probe ready: true, restart count 11
Nov 24 05:39:43.025: INFO: Container node-driver-registrar ready: true, restart count 10
Nov 24 05:39:43.025: INFO: affinity-clusterip-transition-gqrb4 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container affinity-clusterip-transition ready: true, restart count 8
Nov 24 05:39:43.025: INFO: inline-volume-tester-dsr79 started at 2022-11-24 05:27:27 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:39:43.025: INFO: pod-subpath-test-projected-68ws started at 2022-11-24 05:29:23 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container test-container-subpath-projected-68ws ready: false, restart count 0
Nov 24 05:39:43.025: INFO: ss2-0 started at 2022-11-24 05:29:18 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 0
Nov 24 05:39:43.025: INFO: sysctl-99b91704-a59b-41b5-9b20-83e75bec93f2 started at 2022-11-24 05:13:47 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container test-container ready: false, restart count 0
Nov 24 05:39:43.025: INFO: test-container-pod started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 1
Nov 24 05:39:43.025: INFO: pod-1 started at 2022-11-24 05:29:08 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container agnhost-container ready: false, restart count 0
Nov 24 05:39:43.025: INFO: netserver-0 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 1
Nov 24 05:39:43.025: INFO: hostexec-bootstrap-e2e-minion-group-gz1x-jcbgl started at 2022-11-24 05:02:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container agnhost-container ready: true, restart count 8
Nov 24 05:39:43.025: INFO: host-test-container-pod started at 2022-11-24 05:22:59 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container agnhost-container ready: true, restart count 4
Nov 24 05:39:43.025: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-attacher ready: true, restart count 2
Nov 24 05:39:43.025: INFO: test-container-pod started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 2
Nov 24 05:39:43.025: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:29:33 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-attacher ready: true, restart count 0
Nov 24 05:39:43.025: INFO: test-rs-nd7pk started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container httpd ready: true, restart count 3
Nov 24 05:39:43.025: INFO: netserver-0 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: false, restart count 5
Nov 24 05:39:43.025: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:13:34 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-attacher ready: true, restart count 8
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: true, restart count 8
Nov 24 05:39:43.025: INFO: Container csi-resizer ready: true, restart count 8
Nov 24 05:39:43.025: INFO: Container csi-snapshotter ready: true, restart count 8
Nov 24 05:39:43.025: INFO: Container hostpath ready: true, restart count 8
Nov 24 05:39:43.025: INFO: Container liveness-probe ready: true, restart count 8
Nov 24 05:39:43.025: INFO: Container node-driver-registrar ready: true, restart count 8
Nov 24 05:39:43.025: INFO: test-ss-0 started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 5
Nov 24 05:39:43.025: INFO: csi-mockplugin-0 started at 2022-11-24 05:29:33 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: true, restart count 2
Nov 24 05:39:43.025: INFO: Container driver-registrar ready: true, restart count 2
Nov 24 05:39:43.025: INFO: Container mock ready: true, restart count 2
Nov 24 05:39:43.025: INFO: slow-terminating-unready-pod-k5dsj started at 2022-11-24 05:29:03 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container slow-terminating-unready-pod ready: false, restart count 0
Nov 24 05:39:43.025: INFO: netserver-0 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: false, restart count 9
Nov 24 05:39:43.025: INFO: kube-proxy-bootstrap-e2e-minion-group-gz1x started at 2022-11-24 04:58:18 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container kube-proxy ready: false, restart count 9
Nov 24 05:39:43.025: INFO: metrics-server-v0.5.2-867b8754b9-6fxbz started at 2022-11-24 04:59:43 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container metrics-server ready: false, restart count 10
Nov 24 05:39:43.025: INFO: Container metrics-server-nanny ready: false, restart count 11
Nov 24 05:39:43.025: INFO: pod-handle-http-request started at 2022-11-24 05:02:33 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container container-handle-http-request ready: true, restart count 10
Nov 24 05:39:43.025: INFO: Container container-handle-https-request ready: true, restart count 10
Nov 24 05:39:43.025: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:54 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-attacher ready: true, restart count 4
Nov 24 05:39:43.025: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:01 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: true, restart count 0
Nov 24 05:39:43.025: INFO: Container driver-registrar ready: true, restart count 0
Nov 24 05:39:43.025: INFO: Container mock ready: true, restart count 0
Nov 24 05:39:43.025: INFO: external-provisioner-rs6wm started at 2022-11-24 05:29:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container nfs-provisioner ready: false, restart count 5
Nov 24 05:39:43.025: INFO: test-container-pod started at 2022-11-24 05:28:40 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 2
Nov 24 05:39:43.025: INFO: inline-volume-tester-zdzgd started at 2022-11-24 05:29:43 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:39:43.025: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:54 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: false, restart count 8
Nov 24 05:39:43.025: INFO: Container driver-registrar ready: true, restart count 9
Nov 24 05:39:43.025: INFO: Container mock ready: false, restart count 8
Nov 24 05:39:43.025: INFO: fail-once-local-hs44m started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container c ready: false, restart count 1
Nov 24 05:39:43.025: INFO: netserver-0 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: false, restart count 6
Nov 24 05:39:43.025: INFO: fail-once-local-2b8gh started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container c ready: false, restart count 1
Nov 24 05:39:43.025: INFO: csi-mockplugin-0 started at 2022-11-24 05:15:00 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: true, restart count 9
Nov 24 05:39:43.025: INFO: Container driver-registrar ready: true, restart count 11
Nov 24 05:39:43.025: INFO: Container mock ready: true, restart count 9
Nov 24 05:39:43.025: INFO: ss2-2 started at 2022-11-24 05:27:14 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 6
Nov 24 05:39:43.025: INFO: external-provisioner-dhzp9 started at 2022-11-24 05:27:36 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container nfs-provisioner ready: true, restart count 6
Nov 24 05:39:43.025: INFO: netserver-0 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 3
Nov 24 05:39:43.025: INFO: httpd started at 2022-11-24 05:29:24 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container httpd ready: true, restart count 2
Nov 24 05:39:43.025: INFO: metadata-proxy-v0.1-f4l4j started at 2022-11-24 04:58:19 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:39:43.025: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:39:43.025: INFO: netserver-0 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 2
Nov 24 05:39:43.025: INFO: pod-tk9n9 started at 2022-11-24 05:02:27 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container agnhost ready: false, restart count 7
Nov 24 05:39:43.025: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:27:27 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container csi-attacher ready: true, restart count 5
Nov 24 05:39:43.025: INFO: Container csi-provisioner ready: true, restart count 5
Nov 24 05:39:43.025: INFO: Container csi-resizer ready: true, restart count 5
Nov 24 05:39:43.025: INFO: Container csi-snapshotter ready: true, restart count 5
Nov 24 05:39:43.025: INFO: Container hostpath ready: true, restart count 5
Nov 24 05:39:43.025: INFO: Container liveness-probe ready: true, restart count 5
Nov 24 05:39:43.025: INFO: Container node-driver-registrar ready: true, restart count 5
Nov 24 05:39:43.025: INFO: konnectivity-agent-4tkcz started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container konnectivity-agent ready: true, restart count 10
Nov 24 05:39:43.025: INFO: liveness-ecce2bb9-da3e-402f-8446-00acbacd1ce3 started at 2022-11-24 05:01:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container agnhost-container ready: true, restart count 7
Nov 24 05:39:43.025: INFO: netserver-0 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.025: INFO: Container webserver ready: true, restart count 2
Nov 24 05:39:43.742: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:39:43.742: INFO:
Logging node info for node bootstrap-e2e-minion-group-phcp
Nov 24 05:39:43.793: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 18220 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-4441":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-4867":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-24 05:35:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:39:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},}
Nov 24 05:39:43.793: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-phcp
Nov 24 05:39:43.837: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp
Nov 24 05:39:43.941: INFO: netserver-1 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: false, restart count 5
Nov 24 05:39:43.941: INFO: netserver-1 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: false, restart count 6
Nov 24 05:39:43.941: INFO: inline-volume-tester-vw8hc started at 2022-11-24 05:01:58 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:39:43.941: INFO: netserver-1 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: true, restart count 3
Nov 24 05:39:43.941: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:28:07 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: true, restart count 2
Nov 24 05:39:43.941: INFO: kube-proxy-bootstrap-e2e-minion-group-phcp started at 2022-11-24 04:58:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container kube-proxy ready: false, restart count 10
Nov 24 05:39:43.941: INFO: kube-dns-autoscaler-5f6455f985-94cxr started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container autoscaler ready: false, restart count 10
Nov 24 05:39:43.941: INFO: csi-mockplugin-0 started at 2022-11-24 05:05:47 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: true, restart count 8
Nov 24 05:39:43.941: INFO: Container driver-registrar ready: true, restart count 8
Nov 24 05:39:43.941: INFO: Container mock ready: true, restart count 8
Nov 24 05:39:43.941: INFO: inline-volume-tester-cbr4h started at 2022-11-24 05:08:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:39:43.941: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: true, restart count 6
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: true, restart count 6
Nov 24 05:39:43.941: INFO: Container csi-resizer ready: true, restart count 6
Nov 24 05:39:43.941: INFO: Container csi-snapshotter ready: true, restart count 6
Nov 24 05:39:43.941: INFO: Container hostpath ready: true, restart count 6
Nov 24 05:39:43.941: INFO: Container liveness-probe ready: true, restart count 6
Nov 24 05:39:43.941: INFO: Container node-driver-registrar ready: true, restart count 6
Nov 24 05:39:43.941: INFO: csi-mockplugin-0 started at 2022-11-24 05:28:07 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: true, restart count 2
Nov 24 05:39:43.941: INFO: Container driver-registrar ready: true, restart count 2
Nov 24 05:39:43.941: INFO: Container mock ready: true, restart count 2
Nov 24 05:39:43.941: INFO: test-rs-8tt4l started at 2022-11-24 05:28:49 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container httpd ready: true, restart count 2
Nov 24 05:39:43.941: INFO: netserver-1 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: true, restart count 9
Nov 24 05:39:43.941: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: false, restart count 9
Nov 24 05:39:43.941: INFO: Container driver-registrar ready: false, restart count 9
Nov 24 05:39:43.941: INFO: Container mock ready: false, restart count 9
Nov 24 05:39:43.941: INFO: nfs-server started at 2022-11-24 05:29:13 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container nfs-server ready: true, restart count 3
Nov 24 05:39:43.941: INFO: metadata-proxy-v0.1-lghrl started at 2022-11-24 04:58:23 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:39:43.941: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:39:43.941: INFO: l7-default-backend-8549d69d99-29vxh started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container default-http-backend ready: true, restart count 0
Nov 24 05:39:43.941: INFO: volume-snapshot-controller-0 started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container volume-snapshot-controller ready: false, restart count 11
Nov 24 05:39:43.941: INFO: inline-volume-tester-htgjr started at 2022-11-24 05:02:36 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:39:43.941: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:30 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container csi-resizer ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container csi-snapshotter ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container hostpath ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container liveness-probe ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container node-driver-registrar ready: true, restart count 10
Nov 24 05:39:43.941: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:08 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: false, restart count 6
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: false, restart count 6
Nov 24 05:39:43.941: INFO: Container csi-resizer ready: false, restart count 6
Nov 24 05:39:43.941: INFO: Container csi-snapshotter ready: false, restart count 6
Nov 24 05:39:43.941: INFO: Container hostpath ready: false, restart count 6
Nov 24 05:39:43.941: INFO: Container liveness-probe ready: false, restart count 6
Nov 24 05:39:43.941: INFO: Container node-driver-registrar ready: true, restart count 7
Nov 24 05:39:43.941: INFO: netserver-1 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: true, restart count 4
Nov 24 05:39:43.941: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:52 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: true, restart count 2
Nov 24 05:39:43.941: INFO: netserver-1 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: true, restart count 8
Nov 24 05:39:43.941: INFO: ss2-1 started at 2022-11-24 05:29:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: true, restart count 3
Nov 24 05:39:43.941: INFO: inline-volume-tester-7nvw4 started at 2022-11-24 05:27:46 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:39:43.941: INFO: ss2-1 started at 2022-11-24 05:27:06 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: false, restart count 5
Nov 24 05:39:43.941: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: false, restart count 7
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: false, restart count 7
Nov 24 05:39:43.941: INFO: Container csi-resizer ready: false, restart count 7
Nov 24 05:39:43.941: INFO: Container csi-snapshotter ready: false, restart count 7
Nov 24 05:39:43.941: INFO: Container hostpath ready: false, restart count 7
Nov 24 05:39:43.941: INFO: Container liveness-probe ready: false, restart count 7
Nov 24 05:39:43.941: INFO: Container node-driver-registrar ready: false, restart count 7
Nov 24 05:39:43.941: INFO: affinity-clusterip-transition-znjbv started at 2022-11-24 05:02:23 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container affinity-clusterip-transition ready: true, restart count 9
Nov 24 05:39:43.941: INFO: nodeport-test-mbqbt started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container nodeport-test ready: true, restart count 4
Nov 24 05:39:43.941: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:33 +0000 UTC (0+4 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container busybox ready: true, restart count 6
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: false, restart count 4
Nov 24 05:39:43.941: INFO: Container driver-registrar ready: false, restart count 5
Nov 24 05:39:43.941: INFO: Container mock ready: false, restart count 5
Nov 24 05:39:43.941: INFO: coredns-6d97d5ddb-s4dqd started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container coredns ready: false, restart count 11
Nov 24 05:39:43.941: INFO: test-container-pod started at 2022-11-24 05:23:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: true, restart count 5
Nov 24 05:39:43.941: INFO: csi-mockplugin-0 started at 2022-11-24 05:02:32 +0000 UTC (0+4 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container busybox ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: true, restart count 10
Nov 24 05:39:43.941: INFO: Container driver-registrar ready: false, restart count 9
Nov 24 05:39:43.941: INFO: Container mock ready: false, restart count 9
Nov 24 05:39:43.941: INFO: netserver-1 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container webserver ready: false, restart count 6
Nov 24 05:39:43.941: INFO: konnectivity-agent-2t55k started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container konnectivity-agent ready: false, restart count 9
Nov 24 05:39:43.941: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:05:48 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: true, restart count 9
Nov 24 05:39:43.941: INFO: hostexec-bootstrap-e2e-minion-group-phcp-6lwvm started at 2022-11-24 05:29:14 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container agnhost-container ready: true, restart count 3
Nov 24 05:39:43.941: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:52 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: true, restart count 2
Nov 24 05:39:43.941: INFO: Container driver-registrar ready: true, restart count 2
Nov 24 05:39:43.941: INFO: Container mock ready: true, restart count 2
Nov 24 05:39:43.941: INFO: pod-server-1 started at 2022-11-24 05:02:22 +0000 UTC (1+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Init container init ready: true, restart count 3
Nov 24 05:39:43.941: INFO: Container agnhost-container ready: false, restart count 10
Nov 24 05:39:43.941: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:35 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container csi-attacher ready: true, restart count 7
Nov 24 05:39:43.941: INFO: Container csi-provisioner ready: true, restart count 7
Nov 24 05:39:43.941: INFO: Container csi-resizer ready: true, restart count 7
Nov 24 05:39:43.941: INFO: Container csi-snapshotter ready: true, restart count 7
Nov 24 05:39:43.941: INFO: Container hostpath ready: true, restart count 7
Nov 24 05:39:43.941: INFO: Container liveness-probe ready: true, restart count 7
Nov 24 05:39:43.941: INFO: Container node-driver-registrar ready: true, restart count 8
Nov 24 05:39:43.941: INFO: pod-9a7c7833-f26a-4018-9047-bf9628109eaf started at 2022-11-24 05:28:16 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:43.941: INFO: Container write-pod ready: false, restart count 0
Nov 24 05:39:44.327: INFO:
Latency metrics for node bootstrap-e2e-minion-group-phcp
Nov 24 05:39:44.327: INFO:
Logging node info for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:39:44.371: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 18155 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-ephemeral-3073":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:39:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:39:44.371: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:39:44.425: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt
Nov 24 05:39:44.568: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-vnr2m started at 2022-11-24 05:28:58 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container agnhost-container ready: false, restart count 5
Nov 24 05:39:44.568: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:26 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-attacher ready: true, restart count 7
Nov 24 05:39:44.568: INFO: metadata-proxy-v0.1-47gbm started at 2022-11-24 04:58:18 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:39:44.568: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:39:44.568: INFO: pod-client started at 2022-11-24 05:01:50 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container pod-client ready: true, restart count 10
Nov 24 05:39:44.568: INFO: netserver-2 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: true, restart count 4
Nov 24 05:39:44.568: INFO: nfs-server started at 2022-11-24 05:29:01 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container nfs-server ready: true, restart count 1
Nov 24 05:39:44.568: INFO: affinity-clusterip-transition-29sd6 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container affinity-clusterip-transition ready: true, restart count 7
Nov 24 05:39:44.568: INFO: downwardapi-volume-899fb232-b5a9-4752-a4bf-e80f2a5924e1 started at 2022-11-24 05:02:33 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container client-container ready: false, restart count 0
Nov 24 05:39:44.568: INFO: ss2-0 started at 2022-11-24 05:27:01 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: true, restart count 1
Nov 24 05:39:44.568: INFO: netserver-2 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: true, restart count 2
Nov 24 05:39:44.568: INFO: netserver-2 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: false, restart count 6
Nov 24 05:39:44.568: INFO: konnectivity-agent-s27mp started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container konnectivity-agent ready: true, restart count 10
Nov 24 05:39:44.568: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-provisioner ready: false, restart count 9
Nov 24 05:39:44.568: INFO: Container driver-registrar ready: false, restart count 9
Nov 24 05:39:44.568: INFO: Container mock ready: false, restart count 9
Nov 24 05:39:44.568: INFO: netserver-2 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: true, restart count 2
Nov 24 05:39:44.568: INFO: kube-proxy-bootstrap-e2e-minion-group-qvgt started at 2022-11-24 04:58:17 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container kube-proxy ready: true, restart count 11
Nov 24 05:39:44.568: INFO: fail-once-local-nf9km started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container c ready: false, restart count 1
Nov 24 05:39:44.568: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-lqgzg started at 2022-11-24 05:29:06 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container agnhost-container ready: true, restart count 2
Nov 24 05:39:44.568: INFO: nodeport-test-p2clc started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container nodeport-test ready: true, restart count 4
Nov 24 05:39:44.568: INFO: netserver-2 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: true, restart count 3
Nov 24 05:39:44.568: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:21 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-attacher ready: true, restart count 6
Nov 24 05:39:44.568: INFO: Container csi-provisioner ready: true, restart count 6
Nov 24 05:39:44.568: INFO: Container csi-resizer ready: true, restart count 6
Nov 24 05:39:44.568: INFO: Container csi-snapshotter ready: true, restart count 6
Nov 24 05:39:44.568: INFO: Container hostpath ready: true, restart count 6
Nov 24 05:39:44.568: INFO: Container liveness-probe ready: true, restart count 6
Nov 24 05:39:44.568: INFO: Container node-driver-registrar ready: true, restart count 6
Nov 24 05:39:44.568: INFO: netserver-2 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: false, restart count 12
Nov 24 05:39:44.568: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:39 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-attacher ready: false, restart count 6
Nov 24 05:39:44.568: INFO: Container csi-provisioner ready: false, restart count 6
Nov 24 05:39:44.568: INFO: Container csi-resizer ready: false, restart count 6
Nov 24 05:39:44.568: INFO: Container csi-snapshotter ready: false, restart count 6
Nov 24 05:39:44.568: INFO: Container hostpath ready: false, restart count 6
Nov 24 05:39:44.568: INFO: Container liveness-probe ready: false, restart count 6
Nov 24 05:39:44.568: INFO: Container node-driver-registrar ready: false, restart count 6
Nov 24 05:39:44.568: INFO: coredns-6d97d5ddb-r28nc started at 2022-11-24 04:58:43 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container coredns ready: false, restart count 11
Nov 24 05:39:44.568: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:14:23 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-attacher ready: true, restart count 8
Nov 24 05:39:44.568: INFO: Container csi-provisioner ready: true, restart count 8
Nov 24 05:39:44.568: INFO: Container csi-resizer ready: true, restart count 8
Nov 24 05:39:44.568: INFO: Container csi-snapshotter ready: true, restart count 8
Nov 24 05:39:44.568: INFO: Container hostpath ready: true, restart count 8
Nov 24 05:39:44.568: INFO: Container liveness-probe ready: true, restart count 8
Nov 24 05:39:44.568: INFO: Container node-driver-registrar ready: true, restart count 8
Nov 24 05:39:44.568: INFO: execpoddrdpm started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container agnhost-container ready: true, restart count 0
Nov 24 05:39:44.568: INFO: test-rs-nr57v started at 2022-11-24 05:28:49 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container httpd ready: true, restart count 3
Nov 24 05:39:44.568: INFO: Container test-rs ready: true, restart count 3
Nov 24 05:39:44.568: INFO: ss2-2 started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: false, restart count 4
Nov 24 05:39:44.568: INFO: pvc-volume-tester-writer-dhhpl started at 2022-11-24 05:04:50 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container volume-tester ready: false, restart count 0
Nov 24 05:39:44.568: INFO: pod-configmaps-c1868fc3-ec52-478d-ad0f-36361860846b started at 2022-11-24 05:26:45 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container agnhost-container ready: false, restart count 0
Nov 24 05:39:44.568: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:57 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-attacher ready: false, restart count 10
Nov 24 05:39:44.568: INFO: Container csi-provisioner ready: false, restart count 10
Nov 24 05:39:44.568: INFO: Container csi-resizer ready: false, restart count 10
Nov 24 05:39:44.568: INFO: Container csi-snapshotter ready: false, restart count 10
Nov 24 05:39:44.568: INFO: Container hostpath ready: false, restart count 10
Nov 24 05:39:44.568: INFO: Container liveness-probe ready: false, restart count 10
Nov 24 05:39:44.568: INFO: Container node-driver-registrar ready: false, restart count 11
Nov 24 05:39:44.568: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:05:19 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-attacher ready: true, restart count 9
Nov 24 05:39:44.568: INFO: Container csi-provisioner ready: true, restart count 9
Nov 24 05:39:44.568: INFO: Container csi-resizer ready: true, restart count 9
Nov 24 05:39:44.568: INFO: Container csi-snapshotter ready: true, restart count 9
Nov 24 05:39:44.568: INFO: Container hostpath ready: true, restart count 9
Nov 24 05:39:44.568: INFO: Container liveness-probe ready: true, restart count 9
Nov 24 05:39:44.568: INFO: Container node-driver-registrar ready: true, restart count 9
Nov 24 05:39:44.568: INFO: external-provisioner-8vddn started at 2022-11-24 05:02:14 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container nfs-provisioner ready: true, restart count 8
Nov 24 05:39:44.568: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:01:27 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-attacher ready: true, restart count 7
Nov 24 05:39:44.568: INFO: fail-once-local-89bj4 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container c ready: false, restart count 1
Nov 24 05:39:44.568: INFO: netserver-2 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container webserver ready: true, restart count 6
Nov 24 05:39:44.568: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:26 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:39:44.568: INFO: Container csi-provisioner ready: false, restart count 6
Nov 24 05:39:44.568: INFO: Container driver-registrar ready: false, restart count 7
Nov 24 05:39:44.568: INFO: Container mock ready: false, restart count 6
Nov 24 05:39:44.924: INFO:
Latency metrics for node bootstrap-e2e-minion-group-qvgt
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
STEP: Destroying namespace "webhook-1745" for this suite. 11/24/22 05:39:44.924
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\smutate\sconfigmap\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001131e00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
----------
[FAILED]
Nov 24 05:21:59.898: failed to list events in namespace "webhook-4741": Get "https://34.83.136.178/api/v1/namespaces/webhook-4741/events": dial tcp 34.83.136.178:443: connect: connection refused
In [DeferCleanup (Each)] at: test/e2e/framework/debug/dump.go:44
----------
[FAILED]
Nov 24 05:21:59.938: Couldn't delete ns: "webhook-4741": Delete "https://34.83.136.178/api/v1/namespaces/webhook-4741": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-4741", Err:(*net.OpError)(0xc0023c2550)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:19:15.975
Nov 24 05:19:15.975: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:19:15.977
Nov 24 05:19:16.016: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:18.057: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:20.057: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:22.057: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:24.056: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:26.056: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:28.056: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:30.057: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:21:59.818: INFO: Unexpected error:
<*fmt.wrapError | 0xc001f7e200>: {
msg: "wait for service account \"default\" in namespace \"webhook-4741\": timed out waiting for the condition",
err: <*errors.errorString | 0xc000215d80>{
s: "timed out waiting for the condition",
},
}
Nov 24 05:21:59.818: FAIL: wait for service account "default" in namespace "webhook-4741": timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001131e00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:21:59.818: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:21:59.858
STEP: Collecting events from namespace "webhook-4741". 11/24/22 05:21:59.858
Nov 24 05:21:59.897: INFO: Unexpected error: failed to list events in namespace "webhook-4741":
<*url.Error | 0xc002d9a5a0>: {
Op: "Get",
URL: "https://34.83.136.178/api/v1/namespaces/webhook-4741/events",
Err: <*net.OpError | 0xc0048b74a0>{
Op: "dial",
Net: "tcp",
Source: nil,
Addr: <*net.TCPAddr | 0xc0050a8870>{
IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 34, 83, 136, 178],
Port: 443,
Zone: "",
},
Err: <*os.SyscallError | 0xc001f7ef60>{
Syscall: "connect",
Err: <syscall.Errno>0x6f,
},
},
}
Nov 24 05:21:59.898: FAIL: failed to list events in namespace "webhook-4741": Get "https://34.83.136.178/api/v1/namespaces/webhook-4741/events": dial tcp 34.83.136.178:443: connect: connection refused
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/debug.dumpEventsInNamespace(0xc00139c5c0, {0xc004543390, 0xc})
test/e2e/framework/debug/dump.go:44 +0x191
k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc004393040}, {0xc004543390, 0xc})
test/e2e/framework/debug/dump.go:62 +0x8d
k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc00139c650?, {0xc004543390?, 0x7fa7740?})
test/e2e/framework/debug/init/init.go:34 +0x32
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
test/e2e/framework/framework.go:274 +0x6d
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc001131e00)
test/e2e/framework/framework.go:271 +0x179
reflect.Value.call({0x6627cc0?, 0xc001030e20?, 0xc00426efb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0xc0033b0a48?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc001030e20?, 0x29449fc?}, {0xae73300?, 0xc00426ef80?, 0xc0045105b8?})
/usr/local/go/src/reflect/value.go:368 +0xbc
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
STEP: Destroying namespace "webhook-4741" for this suite. 11/24/22 05:21:59.898
Nov 24 05:21:59.938: FAIL: Couldn't delete ns: "webhook-4741": Delete "https://34.83.136.178/api/v1/namespaces/webhook-4741": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-4741", Err:(*net.OpError)(0xc0023c2550)})
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc001131e00)
test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc001030d80?, 0x66e0100?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc001030d80?, 0xc000508f68?}, {0xae73300?, 0x801de88?, 0xc004393040?})
/usr/local/go/src/reflect/value.go:368 +0xbc
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\smutate\scustom\sresource\s\[Conformance\]$'
test/e2e/apimachinery/webhook.go:1808
k8s.io/kubernetes/test/e2e/apimachinery.registerMutatingWebhookForCustomResource(0xc000f83e00, {0xc002608620, 0xc}, 0xc00350a0f0, 0xc00135a380, 0x2742480?)
test/e2e/apimachinery/webhook.go:1808 +0xd51
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.11()
test/e2e/apimachinery/webhook.go:297 +0xad
There were additional failures detected after the initial failure:
[FAILED]
Nov 24 05:29:49.833: Couldn't delete ns: "webhook-7812": Delete "https://34.83.136.178/api/v1/namespaces/webhook-7812": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-7812", Err:(*net.OpError)(0xc0054a25f0)}),Couldn't delete ns: "webhook-7812-markers": Delete "https://34.83.136.178/api/v1/namespaces/webhook-7812-markers": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-7812-markers", Err:(*net.OpError)(0xc00538c690)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:28:39.063
Nov 24 05:28:39.063: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:28:39.064
STEP: Waiting for a default service account to be provisioned in namespace 11/24/22 05:28:39.258
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/24/22 05:28:39.358
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:90
STEP: Setting up server cert 11/24/22 05:28:39.807
STEP: Create role binding to let webhook read extension-apiserver-authentication 11/24/22 05:28:40.168
Nov 24 05:28:40.253: INFO: role binding webhook-auth-reader already exists
STEP: Deploying the webhook pod 11/24/22 05:28:40.253
STEP: Wait for the deployment to be ready 11/24/22 05:28:40.402
Nov 24 05:28:40.899: INFO: deployment "sample-webhook-deployment" doesn't have the required revision set
Nov 24 05:28:43.061: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:45.115: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:47.136: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:49.133: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:51.160: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:53.126: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:55.161: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:57.138: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 41, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:28:59.248: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"NewReplicaSetAvailable", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" has successfully progressed."}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:01.146: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"NewReplicaSetAvailable", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" has successfully progressed."}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:03.160: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"NewReplicaSetAvailable", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" has successfully progressed."}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:05.144: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"NewReplicaSetAvailable", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" has successfully progressed."}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:07.138: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"NewReplicaSetAvailable", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" has successfully progressed."}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:09.174: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"NewReplicaSetAvailable", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" has successfully progressed."}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 24 05:29:11.254: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 40, 0, time.Local), Reason:"NewReplicaSetAvailable", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" has successfully progressed."}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 24, 5, 28, 57, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
STEP: Deploying the webhook service 11/24/22 05:29:13.109
STEP: Verifying the service has paired with the endpoint 11/24/22 05:29:13.238
Nov 24 05:29:14.239: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
[It] should mutate custom resource [Conformance]
test/e2e/apimachinery/webhook.go:291
Nov 24 05:29:14.306: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Registering the mutating webhook for custom resource e2e-test-webhook-9252-crds.webhook.example.com via the AdmissionRegistration API 11/24/22 05:29:15.023
Nov 24 05:29:16.155: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:16.460: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:16.770: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:17.077: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:17.376: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:17.660: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:18.053: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:18.468: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:18.946: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:19.194: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:19.528: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:19.736: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:20.005: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:20.249: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:20.431: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:20.662: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:20.957: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:21.259: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:21.515: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:21.710: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:21.897: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:22.280: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:22.798: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:23.282: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:23.554: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:23.723: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:24.071: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:24.454: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:24.871: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:25.138: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:25.403: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:25.684: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:25.964: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:26.197: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:26.388: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:26.579: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:26.767: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:26.978: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:27.191: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:27.402: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:27.608: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:27.797: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:28.009: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:28.246: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:28.513: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:28.753: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:28.945: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:29.155: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:29.357: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:29.704: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:30.014: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:30.354: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:30.534: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:30.744: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:30.980: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:31.214: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:31.395: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:31.580: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:31.785: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:32.169: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:32.456: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:32.747: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:32.900: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:33.204: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:33.431: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:33.609: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:33.847: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:34.004: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:34.280: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:34.604: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:34.803: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:35.034: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:35.279: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:35.593: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:35.820: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:36.073: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:36.477: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:36.760: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:36.981: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:37.171: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:37.393: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:37.599: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:37.797: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:37.992: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:38.182: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:38.377: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:38.602: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:38.825: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:38.983: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:39.223: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:39.398: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:39.612: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:39.822: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:40.411: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:40.691: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:40.973: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:41.187: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:41.434: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:41.595: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:41.801: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:41.988: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:42.195: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:42.380: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:42.620: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:42.806: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:43.023: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:43.177: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:43.410: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:43.624: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:43.865: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:44.262: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:44.710: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:45.003: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:45.345: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:45.592: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:45.851: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:46.059: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:46.321: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:46.439: INFO: Waiting for webhook configuration to be ready...
Nov 24 05:29:46.439: INFO: Unexpected error: waiting for webhook configuration to be ready:
<*errors.errorString | 0xc00017fa50>: {
s: "timed out waiting for the condition",
}
Nov 24 05:29:46.439: FAIL: waiting for webhook configuration to be ready: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.registerMutatingWebhookForCustomResource(0xc000f83e00, {0xc002608620, 0xc}, 0xc00350a0f0, 0xc00135a380, 0x2742480?)
test/e2e/apimachinery/webhook.go:1808 +0xd51
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.11()
test/e2e/apimachinery/webhook.go:297 +0xad
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:29:47.148: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:29:47.64
STEP: Collecting events from namespace "webhook-7812". 11/24/22 05:29:47.641
STEP: Found 12 events. 11/24/22 05:29:47.793
Nov 24 05:29:47.793: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: { } Scheduled: Successfully assigned webhook-7812/sample-webhook-deployment-865554f4d9-5fvt4 to bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:40 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-865554f4d9 to 1
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:40 +0000 UTC - event for sample-webhook-deployment-865554f4d9: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-865554f4d9-5fvt4
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:44 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:44 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} Created: Created container sample-webhook
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:45 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} Started: Started container sample-webhook
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:46 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} Killing: Stopping container sample-webhook
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:47 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:48 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.237:8444/readyz": context deadline exceeded
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:53 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.244:8444/readyz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:28:53 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} BackOff: Back-off restarting failed container sample-webhook in pod sample-webhook-deployment-865554f4d9-5fvt4_webhook-7812(5cfec4db-374b-4a52-8cd2-31bac8ea4b22)
Nov 24 05:29:47.793: INFO: At 2022-11-24 05:29:12 +0000 UTC - event for sample-webhook-deployment-865554f4d9-5fvt4: {kubelet bootstrap-e2e-minion-group-gz1x} Unhealthy: Readiness probe failed: Get "https://10.64.1.245:8444/readyz": dial tcp 10.64.1.245:8444: i/o timeout
Nov 24 05:29:47.885: INFO: POD NODE PHASE GRACE CONDITIONS
Nov 24 05:29:47.885: INFO:
Nov 24 05:29:47.964: INFO:
Logging node info for node bootstrap-e2e-master
Nov 24 05:29:48.076: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 15023 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:28:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:48.077: INFO:
Logging kubelet events for node bootstrap-e2e-master
Nov 24 05:29:48.169: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 24 05:29:48.258: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 24 05:29:48.258: INFO:
Logging node info for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:48.323: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 17247 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-3519":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8250":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:27:55 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:29:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:55 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:27:27 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:48.323: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:48.375: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:48.442: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-gz1x: error trying to reach service: No agent available
Nov 24 05:29:48.442: INFO:
Logging node info for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:48.498: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 16972 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-4867":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-8063":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {node-problem-detector Update v1 2022-11-24 05:27:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-24 05:29:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:41 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},}
Nov 24 05:29:48.498: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:48.569: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:48.658: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-phcp: error trying to reach service: No agent available
Nov 24 05:29:48.658: INFO:
Logging node info for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:48.752: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 16923 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-24 05:28:00 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-24 05:29:27 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:27:59 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:29:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:48.753: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:48.826: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:48.935: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-qvgt: error trying to reach service: No agent available
STEP: Collecting events from namespace "webhook-7812-markers". 11/24/22 05:29:48.935
STEP: Found 0 events. 11/24/22 05:29:48.994
Nov 24 05:29:49.043: INFO: POD NODE PHASE GRACE CONDITIONS
Nov 24 05:29:49.043: INFO:
Nov 24 05:29:49.152: INFO:
Logging node info for node bootstrap-e2e-master
Nov 24 05:29:49.228: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 15023 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:28:01 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:28:01 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.228: INFO:
Logging kubelet events for node bootstrap-e2e-master
Nov 24 05:29:49.362: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 24 05:29:49.401: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-master:10250/proxy/pods": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.401: INFO:
Logging node info for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:49.440: INFO: Error getting node info Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-gz1x": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.440: INFO: Node Info: &Node{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{},Allocatable:ResourceList{},Phase:,Conditions:[]NodeCondition{},Addresses:[]NodeAddress{},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:0,},},NodeInfo:NodeSystemInfo{MachineID:,SystemUUID:,BootID:,KernelVersion:,OSImage:,ContainerRuntimeVersion:,KubeletVersion:,KubeProxyVersion:,OperatingSystem:,Architecture:,},Images:[]ContainerImage{},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.441: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:49.480: INFO: Unexpected error retrieving node events Get "https://34.83.136.178/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.kind%3DNode%2CinvolvedObject.namespace%3D%2Csource%3Dkubelet%2CinvolvedObject.name%3Dbootstrap-e2e-minion-group-gz1x": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.480: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x
Nov 24 05:29:49.519: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-gz1x: Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-gz1x:10250/proxy/pods": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.519: INFO:
Logging node info for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:49.558: INFO: Error getting node info Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-phcp": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.558: INFO: Node Info: &Node{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{},Allocatable:ResourceList{},Phase:,Conditions:[]NodeCondition{},Addresses:[]NodeAddress{},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:0,},},NodeInfo:NodeSystemInfo{MachineID:,SystemUUID:,BootID:,KernelVersion:,OSImage:,ContainerRuntimeVersion:,KubeletVersion:,KubeProxyVersion:,OperatingSystem:,Architecture:,},Images:[]ContainerImage{},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.558: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:49.597: INFO: Unexpected error retrieving node events Get "https://34.83.136.178/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.name%3Dbootstrap-e2e-minion-group-phcp%2CinvolvedObject.kind%3DNode%2CinvolvedObject.namespace%3D%2Csource%3Dkubelet": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.597: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp
Nov 24 05:29:49.637: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-phcp: Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-phcp:10250/proxy/pods": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.637: INFO:
Logging node info for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:49.676: INFO: Error getting node info Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-qvgt": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.676: INFO: Node Info: &Node{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{},Allocatable:ResourceList{},Phase:,Conditions:[]NodeCondition{},Addresses:[]NodeAddress{},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:0,},},NodeInfo:NodeSystemInfo{MachineID:,SystemUUID:,BootID:,KernelVersion:,OSImage:,ContainerRuntimeVersion:,KubeletVersion:,KubeProxyVersion:,OperatingSystem:,Architecture:,},Images:[]ContainerImage{},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:29:49.676: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:49.715: INFO: Unexpected error retrieving node events Get "https://34.83.136.178/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.kind%3DNode%2CinvolvedObject.namespace%3D%2Csource%3Dkubelet%2CinvolvedObject.name%3Dbootstrap-e2e-minion-group-qvgt": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:29:49.715: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt
Nov 24 05:29:49.754: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-qvgt: Get "https://34.83.136.178/api/v1/nodes/bootstrap-e2e-minion-group-qvgt:10250/proxy/pods": dial tcp 34.83.136.178:443: connect: connection refused
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
STEP: Destroying namespace "webhook-7812" for this suite. 11/24/22 05:29:49.754
STEP: Destroying namespace "webhook-7812-markers" for this suite. 11/24/22 05:29:49.794
Nov 24 05:29:49.833: FAIL: Couldn't delete ns: "webhook-7812": Delete "https://34.83.136.178/api/v1/namespaces/webhook-7812": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-7812", Err:(*net.OpError)(0xc0054a25f0)}),Couldn't delete ns: "webhook-7812-markers": Delete "https://34.83.136.178/api/v1/namespaces/webhook-7812-markers": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/webhook-7812-markers", Err:(*net.OpError)(0xc00538c690)})
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc000f83e00)
test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc001136460?, 0xc001762fb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc001136460?, 0x0?}, {0xae73300?, 0x5?, 0xc002fac660?})
/usr/local/go/src/reflect/value.go:368 +0xbc
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\smutate\scustom\sresource\swith\sdifferent\sstored\sversion\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00047be00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:35:24.443
Nov 24 05:35:24.443: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:35:24.444
Nov 24 05:35:24.484: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:37:29.642: INFO: Unexpected error:
<*fmt.wrapError | 0xc001628000>: {
msg: "wait for service account \"default\" in namespace \"webhook-8585\": timed out waiting for the condition",
err: <*errors.errorString | 0xc000207cc0>{
s: "timed out waiting for the condition",
},
}
Nov 24 05:37:29.642: FAIL: wait for service account "default" in namespace "webhook-8585": timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00047be00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:37:29.642: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:37:29.727
STEP: Collecting events from namespace "webhook-8585". 11/24/22 05:37:29.727
STEP: Found 0 events. 11/24/22 05:37:29.769
Nov 24 05:37:29.810: INFO: POD NODE PHASE GRACE CONDITIONS
Nov 24 05:37:29.810: INFO:
Nov 24 05:37:29.854: INFO:
Logging node info for node bootstrap-e2e-master
Nov 24 05:37:29.895: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 17584 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:35:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:37:29.896: INFO:
Logging kubelet events for node bootstrap-e2e-master
Nov 24 05:37:29.972: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 24 05:37:30.238: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container etcd-container ready: true, restart count 3
Nov 24 05:37:30.238: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container kube-apiserver ready: true, restart count 5
Nov 24 05:37:30.238: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-24 04:57:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container kube-addon-manager ready: true, restart count 5
Nov 24 05:37:30.238: INFO: metadata-proxy-v0.1-ww2rj started at 2022-11-24 04:58:45 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:37:30.238: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:37:30.238: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container kube-controller-manager ready: false, restart count 9
Nov 24 05:37:30.238: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container kube-scheduler ready: true, restart count 7
Nov 24 05:37:30.238: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container etcd-container ready: true, restart count 3
Nov 24 05:37:30.238: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container konnectivity-server-container ready: true, restart count 6
Nov 24 05:37:30.238: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-24 04:57:52 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:30.238: INFO: Container l7-lb-controller ready: false, restart count 9
Nov 24 05:37:30.528: INFO:
Latency metrics for node bootstrap-e2e-master
Nov 24 05:37:30.528: INFO:
Logging node info for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:37:30.658: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 17879 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-ephemeral-9642":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-volume-expand-3803":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8250":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8355":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:37:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:37:30.658: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:37:30.844: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x
Nov 24 05:37:31.949: INFO: fail-once-local-hs44m started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container c ready: false, restart count 1
Nov 24 05:37:31.949: INFO: netserver-0 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: false, restart count 6
Nov 24 05:37:31.949: INFO: netserver-0 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 3
Nov 24 05:37:31.949: INFO: fail-once-local-2b8gh started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container c ready: false, restart count 1
Nov 24 05:37:31.949: INFO: csi-mockplugin-0 started at 2022-11-24 05:15:00 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: true, restart count 9
Nov 24 05:37:31.949: INFO: Container driver-registrar ready: true, restart count 11
Nov 24 05:37:31.949: INFO: Container mock ready: true, restart count 9
Nov 24 05:37:31.949: INFO: ss2-2 started at 2022-11-24 05:27:14 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 6
Nov 24 05:37:31.949: INFO: external-provisioner-dhzp9 started at 2022-11-24 05:27:36 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container nfs-provisioner ready: false, restart count 5
Nov 24 05:37:31.949: INFO: httpd started at 2022-11-24 05:29:24 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container httpd ready: false, restart count 2
Nov 24 05:37:31.949: INFO: metadata-proxy-v0.1-f4l4j started at 2022-11-24 04:58:19 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:37:31.949: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:37:31.949: INFO: netserver-0 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 1
Nov 24 05:37:31.949: INFO: pod-tk9n9 started at 2022-11-24 05:02:27 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container agnhost ready: false, restart count 7
Nov 24 05:37:31.949: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:27:27 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-attacher ready: true, restart count 5
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: true, restart count 5
Nov 24 05:37:31.949: INFO: Container csi-resizer ready: true, restart count 5
Nov 24 05:37:31.949: INFO: Container csi-snapshotter ready: true, restart count 5
Nov 24 05:37:31.949: INFO: Container hostpath ready: true, restart count 5
Nov 24 05:37:31.949: INFO: Container liveness-probe ready: true, restart count 5
Nov 24 05:37:31.949: INFO: Container node-driver-registrar ready: true, restart count 5
Nov 24 05:37:31.949: INFO: konnectivity-agent-4tkcz started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container konnectivity-agent ready: false, restart count 9
Nov 24 05:37:31.949: INFO: liveness-ecce2bb9-da3e-402f-8446-00acbacd1ce3 started at 2022-11-24 05:01:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container agnhost-container ready: false, restart count 6
Nov 24 05:37:31.949: INFO: netserver-0 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 2
Nov 24 05:37:31.949: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:51 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: true, restart count 5
Nov 24 05:37:31.949: INFO: Container driver-registrar ready: true, restart count 6
Nov 24 05:37:31.949: INFO: Container mock ready: true, restart count 5
Nov 24 05:37:31.949: INFO: replace-27821129-fb7zz started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container c ready: false, restart count 0
Nov 24 05:37:31.949: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:12 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-attacher ready: true, restart count 11
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: true, restart count 11
Nov 24 05:37:31.949: INFO: Container csi-resizer ready: true, restart count 11
Nov 24 05:37:31.949: INFO: Container csi-snapshotter ready: true, restart count 11
Nov 24 05:37:31.949: INFO: Container hostpath ready: true, restart count 11
Nov 24 05:37:31.949: INFO: Container liveness-probe ready: true, restart count 11
Nov 24 05:37:31.949: INFO: Container node-driver-registrar ready: true, restart count 10
Nov 24 05:37:31.949: INFO: affinity-clusterip-transition-gqrb4 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container affinity-clusterip-transition ready: true, restart count 8
Nov 24 05:37:31.949: INFO: inline-volume-tester-dsr79 started at 2022-11-24 05:27:27 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:37:31.949: INFO: pod-subpath-test-projected-68ws started at 2022-11-24 05:29:23 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container test-container-subpath-projected-68ws ready: false, restart count 0
Nov 24 05:37:31.949: INFO: ss2-0 started at 2022-11-24 05:29:18 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 0
Nov 24 05:37:31.949: INFO: sysctl-99b91704-a59b-41b5-9b20-83e75bec93f2 started at 2022-11-24 05:13:47 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container test-container ready: false, restart count 0
Nov 24 05:37:31.949: INFO: test-container-pod started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 1
Nov 24 05:37:31.949: INFO: pod-1 started at 2022-11-24 05:29:08 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container agnhost-container ready: false, restart count 0
Nov 24 05:37:31.949: INFO: netserver-0 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 1
Nov 24 05:37:31.949: INFO: test-rs-nd7pk started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container httpd ready: true, restart count 2
Nov 24 05:37:31.949: INFO: hostexec-bootstrap-e2e-minion-group-gz1x-jcbgl started at 2022-11-24 05:02:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container agnhost-container ready: false, restart count 7
Nov 24 05:37:31.949: INFO: host-test-container-pod started at 2022-11-24 05:22:59 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container agnhost-container ready: true, restart count 4
Nov 24 05:37:31.949: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-attacher ready: true, restart count 2
Nov 24 05:37:31.949: INFO: test-container-pod started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 1
Nov 24 05:37:31.949: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:29:33 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-attacher ready: true, restart count 0
Nov 24 05:37:31.949: INFO: netserver-0 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: false, restart count 5
Nov 24 05:37:31.949: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:13:34 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-attacher ready: true, restart count 8
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: true, restart count 8
Nov 24 05:37:31.949: INFO: Container csi-resizer ready: true, restart count 8
Nov 24 05:37:31.949: INFO: Container csi-snapshotter ready: true, restart count 8
Nov 24 05:37:31.949: INFO: Container hostpath ready: true, restart count 8
Nov 24 05:37:31.949: INFO: Container liveness-probe ready: true, restart count 8
Nov 24 05:37:31.949: INFO: Container node-driver-registrar ready: true, restart count 8
Nov 24 05:37:31.949: INFO: test-ss-0 started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 5
Nov 24 05:37:31.949: INFO: csi-mockplugin-0 started at 2022-11-24 05:29:33 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: true, restart count 2
Nov 24 05:37:31.949: INFO: Container driver-registrar ready: true, restart count 2
Nov 24 05:37:31.949: INFO: Container mock ready: true, restart count 2
Nov 24 05:37:31.949: INFO: slow-terminating-unready-pod-k5dsj started at 2022-11-24 05:29:03 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container slow-terminating-unready-pod ready: false, restart count 0
Nov 24 05:37:31.949: INFO: netserver-0 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: false, restart count 9
Nov 24 05:37:31.949: INFO: kube-proxy-bootstrap-e2e-minion-group-gz1x started at 2022-11-24 04:58:18 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container kube-proxy ready: false, restart count 9
Nov 24 05:37:31.949: INFO: metrics-server-v0.5.2-867b8754b9-6fxbz started at 2022-11-24 04:59:43 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container metrics-server ready: false, restart count 9
Nov 24 05:37:31.949: INFO: Container metrics-server-nanny ready: false, restart count 10
Nov 24 05:37:31.949: INFO: pod-handle-http-request started at 2022-11-24 05:02:33 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container container-handle-http-request ready: true, restart count 9
Nov 24 05:37:31.949: INFO: Container container-handle-https-request ready: true, restart count 9
Nov 24 05:37:31.949: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:54 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: false, restart count 8
Nov 24 05:37:31.949: INFO: Container driver-registrar ready: false, restart count 8
Nov 24 05:37:31.949: INFO: Container mock ready: false, restart count 8
Nov 24 05:37:31.949: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:54 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-attacher ready: true, restart count 4
Nov 24 05:37:31.949: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:01 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-provisioner ready: true, restart count 0
Nov 24 05:37:31.949: INFO: Container driver-registrar ready: true, restart count 0
Nov 24 05:37:31.949: INFO: Container mock ready: true, restart count 0
Nov 24 05:37:31.949: INFO: external-provisioner-rs6wm started at 2022-11-24 05:29:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container nfs-provisioner ready: false, restart count 5
Nov 24 05:37:31.949: INFO: test-container-pod started at 2022-11-24 05:28:40 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container webserver ready: true, restart count 0
Nov 24 05:37:31.949: INFO: inline-volume-tester-zdzgd started at 2022-11-24 05:29:43 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:31.949: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:37:33.124: INFO:
Latency metrics for node bootstrap-e2e-minion-group-gz1x
Nov 24 05:37:33.124: INFO:
Logging node info for node bootstrap-e2e-minion-group-phcp
Nov 24 05:37:33.164: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 17725 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-4441":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-4867":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-8063":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-24 05:35:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:36:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},}
Nov 24 05:37:33.165: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-phcp
Nov 24 05:37:33.219: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp
Nov 24 05:37:33.374: INFO: csi-mockplugin-0 started at 2022-11-24 05:28:07 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: true, restart count 0
Nov 24 05:37:33.374: INFO: Container driver-registrar ready: true, restart count 0
Nov 24 05:37:33.374: INFO: Container mock ready: true, restart count 0
Nov 24 05:37:33.374: INFO: test-rs-8tt4l started at 2022-11-24 05:28:49 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container httpd ready: true, restart count 2
Nov 24 05:37:33.374: INFO: netserver-1 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 9
Nov 24 05:37:33.374: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: false, restart count 8
Nov 24 05:37:33.374: INFO: Container driver-registrar ready: true, restart count 9
Nov 24 05:37:33.374: INFO: Container mock ready: false, restart count 8
Nov 24 05:37:33.374: INFO: nfs-server started at 2022-11-24 05:29:13 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container nfs-server ready: true, restart count 3
Nov 24 05:37:33.374: INFO: metadata-proxy-v0.1-lghrl started at 2022-11-24 04:58:23 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:37:33.374: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:37:33.374: INFO: l7-default-backend-8549d69d99-29vxh started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container default-http-backend ready: true, restart count 0
Nov 24 05:37:33.374: INFO: volume-snapshot-controller-0 started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container volume-snapshot-controller ready: false, restart count 11
Nov 24 05:37:33.374: INFO: inline-volume-tester-htgjr started at 2022-11-24 05:02:36 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:37:33.374: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:30 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container csi-resizer ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container csi-snapshotter ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container hostpath ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container liveness-probe ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container node-driver-registrar ready: false, restart count 9
Nov 24 05:37:33.374: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:08 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: false, restart count 6
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: false, restart count 6
Nov 24 05:37:33.374: INFO: Container csi-resizer ready: false, restart count 6
Nov 24 05:37:33.374: INFO: Container csi-snapshotter ready: false, restart count 6
Nov 24 05:37:33.374: INFO: Container hostpath ready: false, restart count 6
Nov 24 05:37:33.374: INFO: Container liveness-probe ready: false, restart count 6
Nov 24 05:37:33.374: INFO: Container node-driver-registrar ready: false, restart count 6
Nov 24 05:37:33.374: INFO: netserver-1 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 4
Nov 24 05:37:33.374: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:52 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: true, restart count 0
Nov 24 05:37:33.374: INFO: netserver-1 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 8
Nov 24 05:37:33.374: INFO: ss2-1 started at 2022-11-24 05:29:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 3
Nov 24 05:37:33.374: INFO: inline-volume-tester-7nvw4 started at 2022-11-24 05:27:46 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:37:33.374: INFO: ss2-1 started at 2022-11-24 05:27:06 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 5
Nov 24 05:37:33.374: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container csi-resizer ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container csi-snapshotter ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container hostpath ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container liveness-probe ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container node-driver-registrar ready: false, restart count 7
Nov 24 05:37:33.374: INFO: affinity-clusterip-transition-znjbv started at 2022-11-24 05:02:23 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container affinity-clusterip-transition ready: false, restart count 8
Nov 24 05:37:33.374: INFO: nodeport-test-mbqbt started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container nodeport-test ready: false, restart count 2
Nov 24 05:37:33.374: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:33 +0000 UTC (0+4 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container busybox ready: false, restart count 5
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: false, restart count 4
Nov 24 05:37:33.374: INFO: Container driver-registrar ready: false, restart count 5
Nov 24 05:37:33.374: INFO: Container mock ready: false, restart count 5
Nov 24 05:37:33.374: INFO: coredns-6d97d5ddb-s4dqd started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container coredns ready: false, restart count 11
Nov 24 05:37:33.374: INFO: test-container-pod started at 2022-11-24 05:23:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 5
Nov 24 05:37:33.374: INFO: csi-mockplugin-0 started at 2022-11-24 05:02:32 +0000 UTC (0+4 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container busybox ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container driver-registrar ready: false, restart count 9
Nov 24 05:37:33.374: INFO: Container mock ready: false, restart count 9
Nov 24 05:37:33.374: INFO: netserver-1 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 6
Nov 24 05:37:33.374: INFO: konnectivity-agent-2t55k started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container konnectivity-agent ready: true, restart count 9
Nov 24 05:37:33.374: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:05:48 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: true, restart count 9
Nov 24 05:37:33.374: INFO: hostexec-bootstrap-e2e-minion-group-phcp-6lwvm started at 2022-11-24 05:29:14 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container agnhost-container ready: false, restart count 2
Nov 24 05:37:33.374: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:52 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: true, restart count 1
Nov 24 05:37:33.374: INFO: Container driver-registrar ready: true, restart count 1
Nov 24 05:37:33.374: INFO: Container mock ready: true, restart count 1
Nov 24 05:37:33.374: INFO: pod-server-1 started at 2022-11-24 05:02:22 +0000 UTC (1+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Init container init ready: true, restart count 2
Nov 24 05:37:33.374: INFO: Container agnhost-container ready: true, restart count 10
Nov 24 05:37:33.374: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:35 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container csi-resizer ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container csi-snapshotter ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container hostpath ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container liveness-probe ready: true, restart count 7
Nov 24 05:37:33.374: INFO: Container node-driver-registrar ready: true, restart count 8
Nov 24 05:37:33.374: INFO: pod-9a7c7833-f26a-4018-9047-bf9628109eaf started at 2022-11-24 05:28:16 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container write-pod ready: false, restart count 0
Nov 24 05:37:33.374: INFO: netserver-1 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: false, restart count 3
Nov 24 05:37:33.374: INFO: netserver-1 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: false, restart count 5
Nov 24 05:37:33.374: INFO: inline-volume-tester-vw8hc started at 2022-11-24 05:01:58 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:37:33.374: INFO: netserver-1 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container webserver ready: true, restart count 3
Nov 24 05:37:33.374: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:28:07 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: true, restart count 2
Nov 24 05:37:33.374: INFO: kube-proxy-bootstrap-e2e-minion-group-phcp started at 2022-11-24 04:58:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container kube-proxy ready: false, restart count 9
Nov 24 05:37:33.374: INFO: kube-dns-autoscaler-5f6455f985-94cxr started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container autoscaler ready: false, restart count 9
Nov 24 05:37:33.374: INFO: csi-mockplugin-0 started at 2022-11-24 05:05:47 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: false, restart count 7
Nov 24 05:37:33.374: INFO: Container driver-registrar ready: false, restart count 7
Nov 24 05:37:33.374: INFO: Container mock ready: false, restart count 7
Nov 24 05:37:33.374: INFO: inline-volume-tester-cbr4h started at 2022-11-24 05:08:35 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-volume-tester ready: false, restart count 0
Nov 24 05:37:33.374: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:33.374: INFO: Container csi-attacher ready: true, restart count 6
Nov 24 05:37:33.374: INFO: Container csi-provisioner ready: true, restart count 6
Nov 24 05:37:33.374: INFO: Container csi-resizer ready: true, restart count 6
Nov 24 05:37:33.374: INFO: Container csi-snapshotter ready: true, restart count 6
Nov 24 05:37:33.374: INFO: Container hostpath ready: true, restart count 6
Nov 24 05:37:33.374: INFO: Container liveness-probe ready: true, restart count 6
Nov 24 05:37:33.374: INFO: Container node-driver-registrar ready: true, restart count 6
Nov 24 05:37:34.175: INFO:
Latency metrics for node bootstrap-e2e-minion-group-phcp
Nov 24 05:37:34.175: INFO:
Logging node info for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:37:34.219: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 17685 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-9601":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 24 05:37:34.220: INFO:
Logging kubelet events for node bootstrap-e2e-minion-group-qvgt
Nov 24 05:37:34.270: INFO:
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt
Nov 24 05:37:34.468: INFO: downwardapi-volume-899fb232-b5a9-4752-a4bf-e80f2a5924e1 started at 2022-11-24 05:02:33 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container client-container ready: false, restart count 0
Nov 24 05:37:34.468: INFO: netserver-2 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: false, restart count 6
Nov 24 05:37:34.468: INFO: konnectivity-agent-s27mp started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container konnectivity-agent ready: false, restart count 9
Nov 24 05:37:34.468: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-provisioner ready: false, restart count 9
Nov 24 05:37:34.468: INFO: Container driver-registrar ready: false, restart count 9
Nov 24 05:37:34.468: INFO: Container mock ready: false, restart count 9
Nov 24 05:37:34.468: INFO: ss2-0 started at 2022-11-24 05:27:01 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 1
Nov 24 05:37:34.468: INFO: netserver-2 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 2
Nov 24 05:37:34.468: INFO: netserver-2 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 2
Nov 24 05:37:34.468: INFO: kube-proxy-bootstrap-e2e-minion-group-qvgt started at 2022-11-24 04:58:17 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container kube-proxy ready: false, restart count 10
Nov 24 05:37:34.468: INFO: fail-once-local-nf9km started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container c ready: false, restart count 1
Nov 24 05:37:34.468: INFO: nodeport-test-p2clc started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container nodeport-test ready: true, restart count 4
Nov 24 05:37:34.468: INFO: netserver-2 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 3
Nov 24 05:37:34.468: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-lqgzg started at 2022-11-24 05:29:06 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container agnhost-container ready: true, restart count 1
Nov 24 05:37:34.468: INFO: netserver-2 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 9
Nov 24 05:37:34.468: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:21 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-attacher ready: true, restart count 5
Nov 24 05:37:34.468: INFO: Container csi-provisioner ready: true, restart count 5
Nov 24 05:37:34.468: INFO: Container csi-resizer ready: true, restart count 5
Nov 24 05:37:34.468: INFO: Container csi-snapshotter ready: true, restart count 5
Nov 24 05:37:34.468: INFO: Container hostpath ready: true, restart count 5
Nov 24 05:37:34.468: INFO: Container liveness-probe ready: true, restart count 5
Nov 24 05:37:34.468: INFO: Container node-driver-registrar ready: true, restart count 5
Nov 24 05:37:34.468: INFO: coredns-6d97d5ddb-r28nc started at 2022-11-24 04:58:43 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container coredns ready: false, restart count 11
Nov 24 05:37:34.468: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:14:23 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-attacher ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container csi-provisioner ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container csi-resizer ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container csi-snapshotter ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container hostpath ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container liveness-probe ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container node-driver-registrar ready: false, restart count 6
Nov 24 05:37:34.468: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:39 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-attacher ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container csi-provisioner ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container csi-resizer ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container csi-snapshotter ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container hostpath ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container liveness-probe ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container node-driver-registrar ready: false, restart count 6
Nov 24 05:37:34.468: INFO: execpoddrdpm started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container agnhost-container ready: true, restart count 0
Nov 24 05:37:34.468: INFO: pvc-volume-tester-writer-dhhpl started at 2022-11-24 05:04:50 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container volume-tester ready: false, restart count 0
Nov 24 05:37:34.468: INFO: pod-configmaps-c1868fc3-ec52-478d-ad0f-36361860846b started at 2022-11-24 05:26:45 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container agnhost-container ready: false, restart count 0
Nov 24 05:37:34.468: INFO: test-rs-nr57v started at 2022-11-24 05:28:49 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container httpd ready: true, restart count 3
Nov 24 05:37:34.468: INFO: Container test-rs ready: true, restart count 3
Nov 24 05:37:34.468: INFO: ss2-2 started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 4
Nov 24 05:37:34.468: INFO: external-provisioner-8vddn started at 2022-11-24 05:02:14 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container nfs-provisioner ready: true, restart count 8
Nov 24 05:37:34.468: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:01:27 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-attacher ready: true, restart count 7
Nov 24 05:37:34.468: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:57 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-attacher ready: true, restart count 10
Nov 24 05:37:34.468: INFO: Container csi-provisioner ready: true, restart count 10
Nov 24 05:37:34.468: INFO: Container csi-resizer ready: true, restart count 10
Nov 24 05:37:34.468: INFO: Container csi-snapshotter ready: true, restart count 10
Nov 24 05:37:34.468: INFO: Container hostpath ready: true, restart count 10
Nov 24 05:37:34.468: INFO: Container liveness-probe ready: true, restart count 10
Nov 24 05:37:34.468: INFO: Container node-driver-registrar ready: true, restart count 11
Nov 24 05:37:34.468: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:05:19 +0000 UTC (0+7 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-attacher ready: false, restart count 8
Nov 24 05:37:34.468: INFO: Container csi-provisioner ready: false, restart count 8
Nov 24 05:37:34.468: INFO: Container csi-resizer ready: false, restart count 8
Nov 24 05:37:34.468: INFO: Container csi-snapshotter ready: false, restart count 8
Nov 24 05:37:34.468: INFO: Container hostpath ready: false, restart count 8
Nov 24 05:37:34.468: INFO: Container liveness-probe ready: false, restart count 8
Nov 24 05:37:34.468: INFO: Container node-driver-registrar ready: false, restart count 8
Nov 24 05:37:34.468: INFO: fail-once-local-89bj4 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container c ready: false, restart count 1
Nov 24 05:37:34.468: INFO: netserver-2 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 5
Nov 24 05:37:34.468: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:26 +0000 UTC (0+3 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-provisioner ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container driver-registrar ready: false, restart count 6
Nov 24 05:37:34.468: INFO: Container mock ready: false, restart count 6
Nov 24 05:37:34.468: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:26 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container csi-attacher ready: false, restart count 6
Nov 24 05:37:34.468: INFO: metadata-proxy-v0.1-47gbm started at 2022-11-24 04:58:18 +0000 UTC (0+2 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container metadata-proxy ready: true, restart count 0
Nov 24 05:37:34.468: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 24 05:37:34.468: INFO: pod-client started at 2022-11-24 05:01:50 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container pod-client ready: true, restart count 10
Nov 24 05:37:34.468: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-vnr2m started at 2022-11-24 05:28:58 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container agnhost-container ready: false, restart count 5
Nov 24 05:37:34.468: INFO: netserver-2 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container webserver ready: true, restart count 4
Nov 24 05:37:34.468: INFO: nfs-server started at 2022-11-24 05:29:01 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container nfs-server ready: true, restart count 0
Nov 24 05:37:34.468: INFO: affinity-clusterip-transition-29sd6 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded)
Nov 24 05:37:34.468: INFO: Container affinity-clusterip-transition ready: true, restart count 7
Nov 24 05:37:34.974: INFO:
Latency metrics for node bootstrap-e2e-minion-group-qvgt
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
STEP: Destroying namespace "webhook-8585" for this suite. 11/24/22 05:37:34.974
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\smutate\scustom\sresource\swith\spruning\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00093be00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:22:08.703
Nov 24 05:22:08.703: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:22:08.705
Nov 24 05:22:08.744: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:10.785: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:12.785: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:14.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:16.785: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:18.785: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:20.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:22.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:24.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:26.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:28.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:30.785: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:32.785: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:34.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:36.784: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:38.785: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:38.824: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:38.824: INFO: Unexpected error:
<*errors.errorString | 0xc00017fa10>: {
s: "timed out waiting for the condition",
}
Nov 24 05:22:38.824: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00093be00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:22:38.825: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:22:38.865
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\smutate\spod\sand\sapply\sdefaults\safter\smutation\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0007a3e00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:18:10.27
Nov 24 05:18:10.270: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:18:10.271
Nov 24 05:18:10.311: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:12.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:14.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:16.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:18.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:20.352: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:22.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:24.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:26.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:28.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:30.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:32.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:34.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:36.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:38.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:40.351: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:40.391: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:18:40.391: INFO: Unexpected error:
<*errors.errorString | 0xc000205d10>: {
s: "timed out waiting for the condition",
}
Nov 24 05:18:40.391: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0007a3e00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:18:40.391: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:18:40.431
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\snot\sbe\sable\sto\smutate\sor\sprevent\sdeletion\sof\swebhook\sconfiguration\sobjects\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000651e00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:30:51.45
Nov 24 05:30:51.450: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:30:51.452
Nov 24 05:30:51.491: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:30:53.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:30:55.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:30:57.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:30:59.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:01.532: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:03.532: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:05.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:07.532: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:09.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:11.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:13.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:15.532: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:17.532: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:19.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:21.531: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:21.571: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:31:21.571: INFO: Unexpected error:
<*errors.errorString | 0xc000195d70>: {
s: "timed out waiting for the condition",
}
Nov 24 05:31:21.571: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000651e00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:31:21.571: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:31:21.611
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sunconditionally\sreject\soperations\son\sfail\sclosed\swebhook\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00093fe00)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260
runtime error: invalid memory address or nil pointer dereference
Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
test/e2e/apimachinery/webhook.go:1680 +0x46
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
test/e2e/apimachinery/webhook.go:106 +0x2f
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:32:51.6
Nov 24 05:32:51.600: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/24/22 05:32:51.602
Nov 24 05:32:51.641: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:53.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:55.680: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:57.680: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:32:59.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:01.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:03.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:05.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:07.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:09.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:11.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:13.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:15.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:17.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:19.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:21.681: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:21.720: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:33:21.720: INFO: Unexpected error:
<*errors.errorString | 0xc0001ffa10>: {
s: "timed out waiting for the condition",
}
Nov 24 05:33:21.721: FAIL: timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00093fe00)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:33:21.721: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:33:21.76
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
tear down framework | framework.go:193
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sWatch\s\[Privileged\:ClusterAdmin\]\sCustomResourceDefinition\sWatch\swatch\son\scustom\sresource\sdefinition\sobjects\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001072b40) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourceDefinition Watch [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:22:02.849 Nov 24 05:22:02.850: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename crd-watch 11/24/22 05:22:02.851 Nov 24 05:22:02.891: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:04.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:06.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:08.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:10.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:12.931: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:14.931: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:16.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:18.931: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:20.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:22.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:24.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:26.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:28.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:30.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:32.930: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:32.969: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:22:32.969: INFO: Unexpected error: <*errors.errorString | 0xc00017fa10>: { s: "timed out waiting for the condition", } Nov 24 05:22:32.970: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001072b40) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourceDefinition Watch [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:22:32.970: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition Watch [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:22:33.009 [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition Watch [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sresources\s\[Privileged\:ClusterAdmin\]\sSimple\sCustomResourceDefinition\screating\/deleting\scustom\sresource\sdefinition\sobjects\sworks\s\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0012aad20) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:31:51.965 Nov 24 05:31:51.966: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename custom-resource-definition 11/24/22 05:31:51.967 Nov 24 05:31:52.007: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:31:54.046: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:31:56.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:31:58.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:00.046: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:02.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:04.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:06.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:08.046: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:10.046: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:12.046: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:14.046: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:16.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:18.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:20.046: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:22.047: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:22.086: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:22.086: INFO: Unexpected error: <*errors.errorString | 0xc0002559e0>: { s: "timed out waiting for the condition", } Nov 24 05:32:22.086: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0012aad20) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:32:22.086: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:32:22.126 [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sresources\s\[Privileged\:ClusterAdmin\]\sSimple\sCustomResourceDefinition\sgetting\/updating\/patching\scustom\sresource\sdefinition\sstatus\ssub\-resource\sworks\s\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000f82c30) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:34:51.606 Nov 24 05:34:51.606: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename custom-resource-definition 11/24/22 05:34:51.607 Nov 24 05:34:51.647: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:53.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:55.686: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:57.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:59.686: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:01.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:03.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:05.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:07.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:09.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:11.686: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:13.686: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:15.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:17.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:19.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:21.687: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:21.727: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:21.727: INFO: Unexpected error: <*errors.errorString | 0xc00017fa50>: { s: "timed out waiting for the condition", } Nov 24 05:35:21.727: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000f82c30) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:35:21.727: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:35:21.766 [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sresources\s\[Privileged\:ClusterAdmin\]\sSimple\sCustomResourceDefinition\slisting\scustom\sresource\sdefinition\sobjects\sworks\s\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010fec30)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[FAILED]
Nov 24 05:22:17.533: failed to list events in namespace "custom-resource-definition-7553": Get "https://34.83.136.178/api/v1/namespaces/custom-resource-definition-7553/events": dial tcp 34.83.136.178:443: connect: connection refused
In [DeferCleanup (Each)] at: test/e2e/framework/debug/dump.go:44
----------
[FAILED]
Nov 24 05:22:17.573: Couldn't delete ns: "custom-resource-definition-7553": Delete "https://34.83.136.178/api/v1/namespaces/custom-resource-definition-7553": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/custom-resource-definition-7553", Err:(*net.OpError)(0xc001844af0)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370
[BeforeEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:19:19.127
Nov 24 05:19:19.127: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename custom-resource-definition 11/24/22 05:19:19.129
Nov 24 05:19:19.168: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:21.208: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:23.208: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:25.207: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:27.208: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:29.207: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:31.208: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:17.453: INFO: Unexpected error:
<*fmt.wrapError | 0xc003ed64e0>: {
msg: "wait for service account \"default\" in namespace \"custom-resource-definition-7553\": timed out waiting for the condition",
err: <*errors.errorString | 0xc000207d70>{
s: "timed out waiting for the condition",
},
}
Nov 24 05:22:17.453: FAIL: wait for service account "default" in namespace "custom-resource-definition-7553": timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010fec30)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:22:17.454: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:22:17.493
STEP: Collecting events from namespace "custom-resource-definition-7553". 11/24/22 05:22:17.493
Nov 24 05:22:17.533: INFO: Unexpected error: failed to list events in namespace "custom-resource-definition-7553":
<*url.Error | 0xc002118000>: {
Op: "Get",
URL: "https://34.83.136.178/api/v1/namespaces/custom-resource-definition-7553/events",
Err: <*net.OpError | 0xc0014924b0>{
Op: "dial",
Net: "tcp",
Source: nil,
Addr: <*net.TCPAddr | 0xc0022b0ea0>{
IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 34, 83, 136, 178],
Port: 443,
Zone: "",
},
Err: <*os.SyscallError | 0xc00411e000>{
Syscall: "connect",
Err: <syscall.Errno>0x6f,
},
},
}
Nov 24 05:22:17.533: FAIL: failed to list events in namespace "custom-resource-definition-7553": Get "https://34.83.136.178/api/v1/namespaces/custom-resource-definition-7553/events": dial tcp 34.83.136.178:443: connect: connection refused
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/debug.dumpEventsInNamespace(0xc000dca5c0, {0xc00078e700, 0x1f})
test/e2e/framework/debug/dump.go:44 +0x191
k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc003f824e0}, {0xc00078e700, 0x1f})
test/e2e/framework/debug/dump.go:62 +0x8d
k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc000dca650?, {0xc00078e700?, 0x7fa7740?})
test/e2e/framework/debug/init/init.go:34 +0x32
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
test/e2e/framework/framework.go:274 +0x6d
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0010fec30)
test/e2e/framework/framework.go:271 +0x179
reflect.Value.call({0x6627cc0?, 0xc001477a30?, 0x2c7d7d7b3a226570?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x7265766f63736944?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc001477a30?, 0x733a66222c7d7b3a?}, {0xae73300?, 0x7079743a66222c7d?, 0x222c7d7d7b3a2265?})
/usr/local/go/src/reflect/value.go:368 +0xbc
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
tear down framework | framework.go:193
STEP: Destroying namespace "custom-resource-definition-7553" for this suite. 11/24/22 05:22:17.533
Nov 24 05:22:17.573: FAIL: Couldn't delete ns: "custom-resource-definition-7553": Delete "https://34.83.136.178/api/v1/namespaces/custom-resource-definition-7553": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/custom-resource-definition-7553", Err:(*net.OpError)(0xc001844af0)})
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc0010fec30)
test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc001477810?, 0x0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc001477810?, 0x7fc8e48?}, {0xae73300?, 0xc00140e000?, 0xc00142a758?})
/usr/local/go/src/reflect/value.go:368 +0xbc
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sresources\s\[Privileged\:ClusterAdmin\]\scustom\sresource\sdefaulting\sfor\srequests\sand\sfrom\sstorage\sworks\s\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0011fbc20) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:32:22.285 Nov 24 05:32:22.285: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename custom-resource-definition 11/24/22 05:32:22.286 Nov 24 05:32:22.326: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:24.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:26.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:28.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:30.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:32.365: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:34.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:36.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:38.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:40.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:42.365: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:44.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:46.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:48.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:50.366: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:52.365: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:52.405: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:52.405: INFO: Unexpected error: <*errors.errorString | 0xc00011dd80>: { s: "timed out waiting for the condition", } Nov 24 05:32:52.405: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0011fbc20) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:32:52.405: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:32:52.445 [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sresources\s\[Privileged\:ClusterAdmin\]\sshould\sinclude\scustom\sresource\sdefinition\sresources\sin\sdiscovery\sdocuments\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0008a8c30) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:30:21.546 Nov 24 05:30:21.546: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename custom-resource-definition 11/24/22 05:30:21.547 Nov 24 05:30:21.586: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:23.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:25.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:27.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:29.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:31.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:33.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:35.627: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:37.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:39.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:41.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:43.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:45.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:47.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:49.627: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:51.626: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:51.665: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:30:51.665: INFO: Unexpected error: <*errors.errorString | 0xc0001cb910>: { s: "timed out waiting for the condition", } Nov 24 05:30:51.665: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0008a8c30) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:30:51.666: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:30:51.705 [DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourcePublishOpenAPI\s\[Privileged\:ClusterAdmin\]\supdates\sthe\spublished\sspec\swhen\sone\sversion\sgets\srenamed\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010f7950) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:34:21.443 Nov 24 05:34:21.443: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename crd-publish-openapi 11/24/22 05:34:21.445 Nov 24 05:34:21.484: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:23.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:25.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:27.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:29.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:31.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:33.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:35.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:37.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:39.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:41.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:43.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:45.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:47.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:49.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:51.524: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:51.563: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:34:51.563: INFO: Unexpected error: <*errors.errorString | 0xc0000d1db0>: { s: "timed out waiting for the condition", } Nov 24 05:34:51.563: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010f7950) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:34:51.564: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:34:51.603 [DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourcePublishOpenAPI\s\[Privileged\:ClusterAdmin\]\sworks\sfor\sCRD\spreserving\sunknown\sfields\sin\san\sembedded\sobject\s\[Conformance\]$'
test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0009fc960)
test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[FAILED]
Nov 24 05:22:08.068: failed to list events in namespace "crd-publish-openapi-4422": Get "https://34.83.136.178/api/v1/namespaces/crd-publish-openapi-4422/events": dial tcp 34.83.136.178:443: connect: connection refused
In [DeferCleanup (Each)] at: test/e2e/framework/debug/dump.go:44
----------
[FAILED]
Nov 24 05:22:08.108: Couldn't delete ns: "crd-publish-openapi-4422": Delete "https://34.83.136.178/api/v1/namespaces/crd-publish-openapi-4422": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/crd-publish-openapi-4422", Err:(*net.OpError)(0xc0021502d0)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370
[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
set up framework | framework.go:178
STEP: Creating a kubernetes client 11/24/22 05:19:10.896
Nov 24 05:19:10.896: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename crd-publish-openapi 11/24/22 05:19:10.898
Nov 24 05:19:10.937: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:12.978: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:14.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:16.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:18.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:20.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:22.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:24.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:26.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:28.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:19:30.977: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused
Nov 24 05:22:07.988: INFO: Unexpected error:
<*fmt.wrapError | 0xc000cf8280>: {
msg: "wait for service account \"default\" in namespace \"crd-publish-openapi-4422\": timed out waiting for the condition",
err: <*errors.errorString | 0xc000243970>{
s: "timed out waiting for the condition",
},
}
Nov 24 05:22:07.988: FAIL: wait for service account "default" in namespace "crd-publish-openapi-4422": timed out waiting for the condition
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0009fc960)
test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
test/e2e/framework/node/init/init.go:32
Nov 24 05:22:07.989: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/24/22 05:22:08.028
STEP: Collecting events from namespace "crd-publish-openapi-4422". 11/24/22 05:22:08.028
Nov 24 05:22:08.068: INFO: Unexpected error: failed to list events in namespace "crd-publish-openapi-4422":
<*url.Error | 0xc002216510>: {
Op: "Get",
URL: "https://34.83.136.178/api/v1/namespaces/crd-publish-openapi-4422/events",
Err: <*net.OpError | 0xc001c9d630>{
Op: "dial",
Net: "tcp",
Source: nil,
Addr: <*net.TCPAddr | 0xc005140570>{
IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 34, 83, 136, 178],
Port: 443,
Zone: "",
},
Err: <*os.SyscallError | 0xc0011e0360>{
Syscall: "connect",
Err: <syscall.Errno>0x6f,
},
},
}
Nov 24 05:22:08.068: FAIL: failed to list events in namespace "crd-publish-openapi-4422": Get "https://34.83.136.178/api/v1/namespaces/crd-publish-openapi-4422/events": dial tcp 34.83.136.178:443: connect: connection refused
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/debug.dumpEventsInNamespace(0xc000b8a5c0, {0xc001d57830, 0x18})
test/e2e/framework/debug/dump.go:44 +0x191
k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc000441ba0}, {0xc001d57830, 0x18})
test/e2e/framework/debug/dump.go:62 +0x8d
k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc000b8a650?, {0xc001d57830?, 0x7fa7740?})
test/e2e/framework/debug/init/init.go:34 +0x32
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
test/e2e/framework/framework.go:274 +0x6d
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0009fc960)
test/e2e/framework/framework.go:271 +0x179
reflect.Value.call({0x6627cc0?, 0xc000ad27d0?, 0xc0013995f0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0xc0013b4b40?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc000ad27d0?, 0xc0013c5460?}, {0xae73300?, 0xc000ae4240?, 0xc0013c5b60?})
/usr/local/go/src/reflect/value.go:368 +0xbc
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
tear down framework | framework.go:193
STEP: Destroying namespace "crd-publish-openapi-4422" for this suite. 11/24/22 05:22:08.068
Nov 24 05:22:08.108: FAIL: Couldn't delete ns: "crd-publish-openapi-4422": Delete "https://34.83.136.178/api/v1/namespaces/crd-publish-openapi-4422": dial tcp 34.83.136.178:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://34.83.136.178/api/v1/namespaces/crd-publish-openapi-4422", Err:(*net.OpError)(0xc0021502d0)})
Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc0009fc960)
test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc000ad26e0?, 0xc002680fb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc000ad26e0?, 0x0?}, {0xae73300?, 0x5?, 0xc001d56708?})
/usr/local/go/src/reflect/value.go:368 +0xbc
from junit_01.xml
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourcePublishOpenAPI\s\[Privileged\:ClusterAdmin\]\sworks\sfor\sCRD\swith\svalidation\sschema\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0008a8960) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:35:23.202 Nov 24 05:35:23.202: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename crd-publish-openapi 11/24/22 05:35:23.204 Nov 24 05:35:23.244: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:35:25.284: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:37:29.895: INFO: Unexpected error: <*fmt.wrapError | 0xc001062000>: { msg: "wait for service account \"default\" in namespace \"crd-publish-openapi-5487\": timed out waiting for the condition", err: <*errors.errorString | 0xc0001cb910>{ s: "timed out waiting for the condition", }, } Nov 24 05:37:29.895: FAIL: wait for service account "default" in namespace "crd-publish-openapi-5487": timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0008a8960) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:37:29.896: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:37:29.983 STEP: Collecting events from namespace "crd-publish-openapi-5487". 11/24/22 05:37:29.983 STEP: Found 0 events. 11/24/22 05:37:30.029 Nov 24 05:37:30.144: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 05:37:30.144: INFO: Nov 24 05:37:30.291: INFO: Logging node info for node bootstrap-e2e-master Nov 24 05:37:30.333: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 17584 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:35:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:32 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:37:30.334: INFO: Logging kubelet events for node bootstrap-e2e-master Nov 24 05:37:30.424: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-master Nov 24 05:37:30.783: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container kube-controller-manager ready: false, restart count 9 Nov 24 05:37:30.783: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container kube-scheduler ready: true, restart count 7 Nov 24 05:37:30.783: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container etcd-container ready: true, restart count 3 Nov 24 05:37:30.783: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container konnectivity-server-container ready: true, restart count 6 Nov 24 05:37:30.783: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-24 04:57:52 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container l7-lb-controller ready: false, restart count 9 Nov 24 05:37:30.783: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container etcd-container ready: true, restart count 3 Nov 24 05:37:30.783: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-24 04:57:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container kube-apiserver ready: true, restart count 5 Nov 24 05:37:30.783: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-24 04:57:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:30.783: INFO: Container kube-addon-manager ready: true, restart count 5 Nov 24 05:37:30.783: INFO: metadata-proxy-v0.1-ww2rj started at 2022-11-24 04:58:45 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:30.783: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:30.783: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:31.851: INFO: Latency metrics for node bootstrap-e2e-master Nov 24 05:37:31.851: INFO: Logging node info for node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:31.911: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-gz1x 165ab2e2-c03d-49de-a6c6-9b7206ee7454 17889 0 2022-11-24 04:58:18 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-gz1x kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-gz1x topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1097":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-ephemeral-9642":"bootstrap-e2e-minion-group-gz1x","csi-hostpath-volume-expand-3803":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-2160":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-3519":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8250":"bootstrap-e2e-minion-group-gz1x","csi-mock-csi-mock-volumes-8355":"bootstrap-e2e-minion-group-gz1x"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:13:43 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:37:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-gz1x,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 05:06:57 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:39 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:18 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:28 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:35.230.95.46,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-gz1x.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:152e5d8bb3a6da82c3e5df652efd2fb3,SystemUUID:152e5d8b-b3a6-da82-c3e5-df652efd2fb3,BootID:e292e4b8-0f86-4d4d-9bbd-6cdc2c215ee4,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:4c4bb2ffc261dd79041c1fa8a7a979520b4c08d8e535583e8fbbf22690d13bb1 registry.k8s.io/etcd:3.5.5-1],SizeBytes:102440299,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:37:31.911: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:32.166: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:32.916: INFO: pod-subpath-test-projected-68ws started at 2022-11-24 05:29:23 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container test-container-subpath-projected-68ws ready: false, restart count 0 Nov 24 05:37:32.916: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:12 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-attacher ready: true, restart count 11 Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: true, restart count 11 Nov 24 05:37:32.916: INFO: Container csi-resizer ready: true, restart count 11 Nov 24 05:37:32.916: INFO: Container csi-snapshotter ready: true, restart count 11 Nov 24 05:37:32.916: INFO: Container hostpath ready: true, restart count 11 Nov 24 05:37:32.916: INFO: Container liveness-probe ready: true, restart count 11 Nov 24 05:37:32.916: INFO: Container node-driver-registrar ready: true, restart count 10 Nov 24 05:37:32.916: INFO: affinity-clusterip-transition-gqrb4 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container affinity-clusterip-transition ready: true, restart count 8 Nov 24 05:37:32.916: INFO: inline-volume-tester-dsr79 started at 2022-11-24 05:27:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:32.916: INFO: ss2-0 started at 2022-11-24 05:29:18 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 0 Nov 24 05:37:32.916: INFO: sysctl-99b91704-a59b-41b5-9b20-83e75bec93f2 started at 2022-11-24 05:13:47 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container test-container ready: false, restart count 0 Nov 24 05:37:32.916: INFO: test-container-pod started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:32.916: INFO: pod-1 started at 2022-11-24 05:29:08 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 05:37:32.916: INFO: netserver-0 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:32.916: INFO: test-container-pod started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:32.916: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:29:33 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:37:32.916: INFO: test-rs-nd7pk started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container httpd ready: true, restart count 2 Nov 24 05:37:32.916: INFO: hostexec-bootstrap-e2e-minion-group-gz1x-jcbgl started at 2022-11-24 05:02:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container agnhost-container ready: false, restart count 7 Nov 24 05:37:32.916: INFO: host-test-container-pod started at 2022-11-24 05:22:59 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container agnhost-container ready: true, restart count 4 Nov 24 05:37:32.916: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-attacher ready: true, restart count 2 Nov 24 05:37:32.916: INFO: netserver-0 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: false, restart count 5 Nov 24 05:37:32.916: INFO: csi-mockplugin-0 started at 2022-11-24 05:29:33 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: true, restart count 2 Nov 24 05:37:32.916: INFO: Container driver-registrar ready: true, restart count 2 Nov 24 05:37:32.916: INFO: Container mock ready: true, restart count 2 Nov 24 05:37:32.916: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:13:34 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-attacher ready: true, restart count 8 Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: true, restart count 8 Nov 24 05:37:32.916: INFO: Container csi-resizer ready: true, restart count 8 Nov 24 05:37:32.916: INFO: Container csi-snapshotter ready: true, restart count 8 Nov 24 05:37:32.916: INFO: Container hostpath ready: true, restart count 8 Nov 24 05:37:32.916: INFO: Container liveness-probe ready: true, restart count 8 Nov 24 05:37:32.916: INFO: Container node-driver-registrar ready: true, restart count 8 Nov 24 05:37:32.916: INFO: test-ss-0 started at 2022-11-24 05:28:39 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 5 Nov 24 05:37:32.916: INFO: netserver-0 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: false, restart count 9 Nov 24 05:37:32.916: INFO: slow-terminating-unready-pod-k5dsj started at 2022-11-24 05:29:03 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container slow-terminating-unready-pod ready: false, restart count 0 Nov 24 05:37:32.916: INFO: pod-handle-http-request started at 2022-11-24 05:02:33 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:32.916: INFO: Container container-handle-http-request ready: true, restart count 9 Nov 24 05:37:32.916: INFO: Container container-handle-https-request ready: true, restart count 9 Nov 24 05:37:32.916: INFO: kube-proxy-bootstrap-e2e-minion-group-gz1x started at 2022-11-24 04:58:18 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container kube-proxy ready: false, restart count 9 Nov 24 05:37:32.916: INFO: metrics-server-v0.5.2-867b8754b9-6fxbz started at 2022-11-24 04:59:43 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:32.916: INFO: Container metrics-server ready: false, restart count 9 Nov 24 05:37:32.916: INFO: Container metrics-server-nanny ready: false, restart count 10 Nov 24 05:37:32.916: INFO: inline-volume-tester-zdzgd started at 2022-11-24 05:29:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:32.916: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:54 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: false, restart count 8 Nov 24 05:37:32.916: INFO: Container driver-registrar ready: false, restart count 8 Nov 24 05:37:32.916: INFO: Container mock ready: false, restart count 8 Nov 24 05:37:32.916: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:54 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-attacher ready: true, restart count 4 Nov 24 05:37:32.916: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:01 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:37:32.916: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 05:37:32.916: INFO: Container mock ready: true, restart count 0 Nov 24 05:37:32.916: INFO: external-provisioner-rs6wm started at 2022-11-24 05:29:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container nfs-provisioner ready: false, restart count 5 Nov 24 05:37:32.916: INFO: test-container-pod started at 2022-11-24 05:28:40 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 0 Nov 24 05:37:32.916: INFO: fail-once-local-hs44m started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container c ready: false, restart count 1 Nov 24 05:37:32.916: INFO: netserver-0 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: false, restart count 6 Nov 24 05:37:32.916: INFO: ss2-2 started at 2022-11-24 05:27:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 6 Nov 24 05:37:32.916: INFO: external-provisioner-dhzp9 started at 2022-11-24 05:27:36 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container nfs-provisioner ready: false, restart count 5 Nov 24 05:37:32.916: INFO: netserver-0 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:32.916: INFO: fail-once-local-2b8gh started at 2022-11-24 05:04:42 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container c ready: false, restart count 1 Nov 24 05:37:32.916: INFO: csi-mockplugin-0 started at 2022-11-24 05:15:00 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: true, restart count 9 Nov 24 05:37:32.916: INFO: Container driver-registrar ready: true, restart count 11 Nov 24 05:37:32.916: INFO: Container mock ready: true, restart count 9 Nov 24 05:37:32.916: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:27:27 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-attacher ready: true, restart count 5 Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: true, restart count 5 Nov 24 05:37:32.916: INFO: Container csi-resizer ready: true, restart count 5 Nov 24 05:37:32.916: INFO: Container csi-snapshotter ready: true, restart count 5 Nov 24 05:37:32.916: INFO: Container hostpath ready: true, restart count 5 Nov 24 05:37:32.916: INFO: Container liveness-probe ready: true, restart count 5 Nov 24 05:37:32.916: INFO: Container node-driver-registrar ready: true, restart count 5 Nov 24 05:37:32.916: INFO: httpd started at 2022-11-24 05:29:24 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container httpd ready: false, restart count 2 Nov 24 05:37:32.916: INFO: metadata-proxy-v0.1-f4l4j started at 2022-11-24 04:58:19 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:32.916: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:32.916: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:32.916: INFO: netserver-0 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:32.916: INFO: pod-tk9n9 started at 2022-11-24 05:02:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container agnhost ready: false, restart count 7 Nov 24 05:37:32.916: INFO: konnectivity-agent-4tkcz started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container konnectivity-agent ready: false, restart count 9 Nov 24 05:37:32.916: INFO: liveness-ecce2bb9-da3e-402f-8446-00acbacd1ce3 started at 2022-11-24 05:01:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container agnhost-container ready: false, restart count 6 Nov 24 05:37:32.916: INFO: netserver-0 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container webserver ready: true, restart count 2 Nov 24 05:37:32.916: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:51 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:32.916: INFO: Container csi-provisioner ready: true, restart count 5 Nov 24 05:37:32.916: INFO: Container driver-registrar ready: true, restart count 6 Nov 24 05:37:32.916: INFO: Container mock ready: true, restart count 5 Nov 24 05:37:32.916: INFO: replace-27821129-fb7zz started at 2022-11-24 05:29:00 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:32.916: INFO: Container c ready: false, restart count 0 Nov 24 05:37:36.668: INFO: Latency metrics for node bootstrap-e2e-minion-group-gz1x Nov 24 05:37:36.668: INFO: Logging node info for node bootstrap-e2e-minion-group-phcp Nov 24 05:37:36.710: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-phcp 61d942f3-38e1-4ae9-bf2b-ffaef8c1c92e 17725 0 2022-11-24 04:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-phcp kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-phcp topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-4441":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-8468":"bootstrap-e2e-minion-group-phcp","csi-hostpath-ephemeral-9380":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-4761":"bootstrap-e2e-minion-group-phcp","csi-hostpath-volume-expand-5396":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-1230":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-4867":"bootstrap-e2e-minion-group-phcp","csi-mock-csi-mock-volumes-8063":"bootstrap-e2e-minion-group-phcp"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kubelet Update v1 2022-11-24 04:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 05:28:41 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-24 05:35:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:36:12 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-phcp,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 05:07:01 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:38 +0000 UTC,LastTransitionTime:2022-11-24 04:58:25 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:29 +0000 UTC,LastTransitionTime:2022-11-24 04:58:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:35.199.163.34,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-phcp.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:db4f4f14f2dd47aeb148a5e53ba59185,SystemUUID:db4f4f14-f2dd-47ae-b148-a5e53ba59185,BootID:e0961361-1ca6-4dee-8ce0-14dbc97ef9d8,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659 kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8468^1f4c0e6c-6bb5-11ed-8705-aec21468b659,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-6342^cabd367b-6bb8-11ed-87af-ba6bb008dd7b,DevicePath:,},},Config:nil,},} Nov 24 05:37:36.710: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-phcp Nov 24 05:37:36.754: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-phcp Nov 24 05:37:37.131: INFO: inline-volume-tester-vw8hc started at 2022-11-24 05:01:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:37.131: INFO: netserver-1 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:37.131: INFO: kube-proxy-bootstrap-e2e-minion-group-phcp started at 2022-11-24 04:58:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container kube-proxy ready: false, restart count 9 Nov 24 05:37:37.131: INFO: kube-dns-autoscaler-5f6455f985-94cxr started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container autoscaler ready: false, restart count 9 Nov 24 05:37:37.131: INFO: csi-mockplugin-0 started at 2022-11-24 05:05:47 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: false, restart count 7 Nov 24 05:37:37.131: INFO: Container driver-registrar ready: false, restart count 7 Nov 24 05:37:37.131: INFO: Container mock ready: false, restart count 7 Nov 24 05:37:37.131: INFO: inline-volume-tester-cbr4h started at 2022-11-24 05:08:35 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:37.131: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:28:07 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: true, restart count 2 Nov 24 05:37:37.131: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: true, restart count 6 Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: true, restart count 6 Nov 24 05:37:37.131: INFO: Container csi-resizer ready: true, restart count 6 Nov 24 05:37:37.131: INFO: Container csi-snapshotter ready: true, restart count 6 Nov 24 05:37:37.131: INFO: Container hostpath ready: true, restart count 6 Nov 24 05:37:37.131: INFO: Container liveness-probe ready: true, restart count 6 Nov 24 05:37:37.131: INFO: Container node-driver-registrar ready: true, restart count 6 Nov 24 05:37:37.131: INFO: csi-mockplugin-0 started at 2022-11-24 05:28:07 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: true, restart count 0 Nov 24 05:37:37.131: INFO: Container driver-registrar ready: true, restart count 0 Nov 24 05:37:37.131: INFO: Container mock ready: true, restart count 0 Nov 24 05:37:37.131: INFO: test-rs-8tt4l started at 2022-11-24 05:28:49 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container httpd ready: true, restart count 2 Nov 24 05:37:37.131: INFO: netserver-1 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: true, restart count 9 Nov 24 05:37:37.131: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:50 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: false, restart count 8 Nov 24 05:37:37.131: INFO: Container driver-registrar ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container mock ready: false, restart count 8 Nov 24 05:37:37.131: INFO: nfs-server started at 2022-11-24 05:29:13 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container nfs-server ready: true, restart count 3 Nov 24 05:37:37.131: INFO: metadata-proxy-v0.1-lghrl started at 2022-11-24 04:58:23 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:37.131: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:37.131: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:37.131: INFO: l7-default-backend-8549d69d99-29vxh started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container default-http-backend ready: true, restart count 0 Nov 24 05:37:37.131: INFO: volume-snapshot-controller-0 started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container volume-snapshot-controller ready: false, restart count 11 Nov 24 05:37:37.131: INFO: inline-volume-tester-htgjr started at 2022-11-24 05:02:36 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:37.131: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:02:30 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container csi-resizer ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container csi-snapshotter ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container hostpath ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container liveness-probe ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container node-driver-registrar ready: false, restart count 9 Nov 24 05:37:37.131: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:08 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:37.131: INFO: Container csi-resizer ready: false, restart count 6 Nov 24 05:37:37.131: INFO: Container csi-snapshotter ready: false, restart count 6 Nov 24 05:37:37.131: INFO: Container hostpath ready: false, restart count 6 Nov 24 05:37:37.131: INFO: Container liveness-probe ready: false, restart count 6 Nov 24 05:37:37.131: INFO: Container node-driver-registrar ready: false, restart count 6 Nov 24 05:37:37.131: INFO: netserver-1 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: true, restart count 4 Nov 24 05:37:37.131: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:26:52 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: true, restart count 0 Nov 24 05:37:37.131: INFO: netserver-1 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: true, restart count 8 Nov 24 05:37:37.131: INFO: ss2-1 started at 2022-11-24 05:29:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:37.131: INFO: inline-volume-tester-7nvw4 started at 2022-11-24 05:27:46 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-volume-tester ready: false, restart count 0 Nov 24 05:37:37.131: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container csi-resizer ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container csi-snapshotter ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container hostpath ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container liveness-probe ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container node-driver-registrar ready: false, restart count 7 Nov 24 05:37:37.131: INFO: affinity-clusterip-transition-znjbv started at 2022-11-24 05:02:23 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container affinity-clusterip-transition ready: false, restart count 8 Nov 24 05:37:37.131: INFO: nodeport-test-mbqbt started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container nodeport-test ready: false, restart count 2 Nov 24 05:37:37.131: INFO: ss2-1 started at 2022-11-24 05:27:06 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: false, restart count 5 Nov 24 05:37:37.131: INFO: coredns-6d97d5ddb-s4dqd started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container coredns ready: false, restart count 11 Nov 24 05:37:37.131: INFO: test-container-pod started at 2022-11-24 05:23:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: true, restart count 5 Nov 24 05:37:37.131: INFO: csi-mockplugin-0 started at 2022-11-24 05:27:33 +0000 UTC (0+4 container statuses recorded) Nov 24 05:37:37.131: INFO: Container busybox ready: false, restart count 5 Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: false, restart count 4 Nov 24 05:37:37.131: INFO: Container driver-registrar ready: false, restart count 5 Nov 24 05:37:37.131: INFO: Container mock ready: false, restart count 5 Nov 24 05:37:37.131: INFO: csi-mockplugin-0 started at 2022-11-24 05:02:32 +0000 UTC (0+4 container statuses recorded) Nov 24 05:37:37.131: INFO: Container busybox ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container driver-registrar ready: false, restart count 9 Nov 24 05:37:37.131: INFO: Container mock ready: false, restart count 9 Nov 24 05:37:37.131: INFO: netserver-1 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: true, restart count 6 Nov 24 05:37:37.131: INFO: konnectivity-agent-2t55k started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container konnectivity-agent ready: true, restart count 9 Nov 24 05:37:37.131: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:05:48 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: true, restart count 9 Nov 24 05:37:37.131: INFO: csi-mockplugin-0 started at 2022-11-24 05:26:52 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: true, restart count 1 Nov 24 05:37:37.131: INFO: Container driver-registrar ready: true, restart count 1 Nov 24 05:37:37.131: INFO: Container mock ready: true, restart count 1 Nov 24 05:37:37.131: INFO: pod-server-1 started at 2022-11-24 05:02:22 +0000 UTC (1+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Init container init ready: true, restart count 2 Nov 24 05:37:37.131: INFO: Container agnhost-container ready: true, restart count 10 Nov 24 05:37:37.131: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:35 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:37.131: INFO: Container csi-attacher ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container csi-provisioner ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container csi-resizer ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container csi-snapshotter ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container hostpath ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container liveness-probe ready: true, restart count 7 Nov 24 05:37:37.131: INFO: Container node-driver-registrar ready: true, restart count 8 Nov 24 05:37:37.131: INFO: pod-9a7c7833-f26a-4018-9047-bf9628109eaf started at 2022-11-24 05:28:16 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container write-pod ready: false, restart count 0 Nov 24 05:37:37.131: INFO: hostexec-bootstrap-e2e-minion-group-phcp-6lwvm started at 2022-11-24 05:29:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container agnhost-container ready: false, restart count 2 Nov 24 05:37:37.131: INFO: netserver-1 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: false, restart count 3 Nov 24 05:37:37.131: INFO: netserver-1 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:37.131: INFO: Container webserver ready: false, restart count 5 Nov 24 05:37:39.336: INFO: Latency metrics for node bootstrap-e2e-minion-group-phcp Nov 24 05:37:39.336: INFO: Logging node info for node bootstrap-e2e-minion-group-qvgt Nov 24 05:37:39.381: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-qvgt 7490b1bd-0963-4d2f-ad05-1a6ab3d50853 17685 0 2022-11-24 04:58:17 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-qvgt kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-qvgt topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-1981":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-3306":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-provisioning-9601":"bootstrap-e2e-minion-group-qvgt","csi-hostpath-volume-2486":"bootstrap-e2e-minion-group-qvgt","csi-mock-csi-mock-volumes-2173":"bootstrap-e2e-minion-group-qvgt"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:17 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-24 05:28:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-24 05:35:35 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-24 05:35:53 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-minion-group-qvgt,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 05:07:04 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:21 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-24 05:35:35 +0000 UTC,LastTransitionTime:2022-11-24 04:58:20 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:17 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:35:33 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.199.110,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-qvgt.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:df664e20215057ddab4ea720e9bd4453,SystemUUID:df664e20-2150-57dd-ab4e-a720e9bd4453,BootID:52ef6e10-cfa5-496c-a64e-d232bd886d30,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:db048754ae68ae337d8fa96494c96d2a1204c3320f5dcf7e8e71085adec85da6 registry.k8s.io/e2e-test-images/nginx:1.15-4],SizeBytes:7000082,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 24 05:37:39.381: INFO: Logging kubelet events for node bootstrap-e2e-minion-group-qvgt Nov 24 05:37:39.430: INFO: Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-qvgt Nov 24 05:37:39.680: INFO: fail-once-local-nf9km started at 2022-11-24 05:02:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container c ready: false, restart count 1 Nov 24 05:37:39.680: INFO: kube-proxy-bootstrap-e2e-minion-group-qvgt started at 2022-11-24 04:58:17 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container kube-proxy ready: false, restart count 10 Nov 24 05:37:39.680: INFO: netserver-2 started at 2022-11-24 05:27:19 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: true, restart count 3 Nov 24 05:37:39.680: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-lqgzg started at 2022-11-24 05:29:06 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container agnhost-container ready: true, restart count 1 Nov 24 05:37:39.680: INFO: nodeport-test-p2clc started at 2022-11-24 05:26:51 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container nodeport-test ready: true, restart count 4 Nov 24 05:37:39.680: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:08:21 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-attacher ready: true, restart count 5 Nov 24 05:37:39.680: INFO: Container csi-provisioner ready: true, restart count 5 Nov 24 05:37:39.680: INFO: Container csi-resizer ready: true, restart count 5 Nov 24 05:37:39.680: INFO: Container csi-snapshotter ready: true, restart count 5 Nov 24 05:37:39.680: INFO: Container hostpath ready: true, restart count 5 Nov 24 05:37:39.680: INFO: Container liveness-probe ready: true, restart count 5 Nov 24 05:37:39.680: INFO: Container node-driver-registrar ready: true, restart count 5 Nov 24 05:37:39.680: INFO: netserver-2 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: false, restart count 9 Nov 24 05:37:39.680: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:14:23 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container csi-resizer ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container csi-snapshotter ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container hostpath ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container liveness-probe ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container node-driver-registrar ready: false, restart count 6 Nov 24 05:37:39.680: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:28:39 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container csi-resizer ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container csi-snapshotter ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container hostpath ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container liveness-probe ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container node-driver-registrar ready: false, restart count 6 Nov 24 05:37:39.680: INFO: coredns-6d97d5ddb-r28nc started at 2022-11-24 04:58:43 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container coredns ready: false, restart count 11 Nov 24 05:37:39.680: INFO: execpoddrdpm started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container agnhost-container ready: true, restart count 0 Nov 24 05:37:39.680: INFO: pod-configmaps-c1868fc3-ec52-478d-ad0f-36361860846b started at 2022-11-24 05:26:45 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container agnhost-container ready: false, restart count 0 Nov 24 05:37:39.680: INFO: test-rs-nr57v started at 2022-11-24 05:28:49 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:39.680: INFO: Container httpd ready: true, restart count 3 Nov 24 05:37:39.680: INFO: Container test-rs ready: true, restart count 3 Nov 24 05:37:39.680: INFO: ss2-2 started at 2022-11-24 05:29:28 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: true, restart count 4 Nov 24 05:37:39.680: INFO: pvc-volume-tester-writer-dhhpl started at 2022-11-24 05:04:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container volume-tester ready: false, restart count 0 Nov 24 05:37:39.680: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:01:27 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-attacher ready: true, restart count 7 Nov 24 05:37:39.680: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:01:57 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-attacher ready: true, restart count 10 Nov 24 05:37:39.680: INFO: Container csi-provisioner ready: true, restart count 10 Nov 24 05:37:39.680: INFO: Container csi-resizer ready: true, restart count 10 Nov 24 05:37:39.680: INFO: Container csi-snapshotter ready: true, restart count 10 Nov 24 05:37:39.680: INFO: Container hostpath ready: true, restart count 10 Nov 24 05:37:39.680: INFO: Container liveness-probe ready: true, restart count 10 Nov 24 05:37:39.680: INFO: Container node-driver-registrar ready: true, restart count 11 Nov 24 05:37:39.680: INFO: csi-hostpathplugin-0 started at 2022-11-24 05:05:19 +0000 UTC (0+7 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-attacher ready: false, restart count 8 Nov 24 05:37:39.680: INFO: Container csi-provisioner ready: false, restart count 8 Nov 24 05:37:39.680: INFO: Container csi-resizer ready: false, restart count 8 Nov 24 05:37:39.680: INFO: Container csi-snapshotter ready: false, restart count 8 Nov 24 05:37:39.680: INFO: Container hostpath ready: false, restart count 8 Nov 24 05:37:39.680: INFO: Container liveness-probe ready: false, restart count 8 Nov 24 05:37:39.680: INFO: Container node-driver-registrar ready: false, restart count 8 Nov 24 05:37:39.680: INFO: external-provisioner-8vddn started at 2022-11-24 05:02:14 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container nfs-provisioner ready: true, restart count 8 Nov 24 05:37:39.680: INFO: netserver-2 started at 2022-11-24 05:14:25 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: true, restart count 5 Nov 24 05:37:39.680: INFO: fail-once-local-89bj4 started at 2022-11-24 05:02:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container c ready: false, restart count 1 Nov 24 05:37:39.680: INFO: csi-mockplugin-0 started at 2022-11-24 05:12:26 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-provisioner ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container driver-registrar ready: false, restart count 6 Nov 24 05:37:39.680: INFO: Container mock ready: false, restart count 6 Nov 24 05:37:39.680: INFO: pod-client started at 2022-11-24 05:01:50 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container pod-client ready: true, restart count 10 Nov 24 05:37:39.680: INFO: hostexec-bootstrap-e2e-minion-group-qvgt-vnr2m started at 2022-11-24 05:28:58 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container agnhost-container ready: false, restart count 5 Nov 24 05:37:39.680: INFO: csi-mockplugin-attacher-0 started at 2022-11-24 05:12:26 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-attacher ready: false, restart count 6 Nov 24 05:37:39.680: INFO: metadata-proxy-v0.1-47gbm started at 2022-11-24 04:58:18 +0000 UTC (0+2 container statuses recorded) Nov 24 05:37:39.680: INFO: Container metadata-proxy ready: true, restart count 0 Nov 24 05:37:39.680: INFO: Container prometheus-to-sd-exporter ready: true, restart count 0 Nov 24 05:37:39.680: INFO: netserver-2 started at 2022-11-24 05:26:57 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: true, restart count 4 Nov 24 05:37:39.680: INFO: nfs-server started at 2022-11-24 05:29:01 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container nfs-server ready: true, restart count 0 Nov 24 05:37:39.680: INFO: affinity-clusterip-transition-29sd6 started at 2022-11-24 05:02:22 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container affinity-clusterip-transition ready: true, restart count 7 Nov 24 05:37:39.680: INFO: downwardapi-volume-899fb232-b5a9-4752-a4bf-e80f2a5924e1 started at 2022-11-24 05:02:33 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container client-container ready: false, restart count 0 Nov 24 05:37:39.680: INFO: csi-mockplugin-0 started at 2022-11-24 05:01:27 +0000 UTC (0+3 container statuses recorded) Nov 24 05:37:39.680: INFO: Container csi-provisioner ready: false, restart count 9 Nov 24 05:37:39.680: INFO: Container driver-registrar ready: false, restart count 9 Nov 24 05:37:39.680: INFO: Container mock ready: false, restart count 9 Nov 24 05:37:39.680: INFO: ss2-0 started at 2022-11-24 05:27:01 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: true, restart count 1 Nov 24 05:37:39.680: INFO: netserver-2 started at 2022-11-24 05:28:15 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: true, restart count 2 Nov 24 05:37:39.680: INFO: netserver-2 started at 2022-11-24 05:29:04 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: false, restart count 6 Nov 24 05:37:39.680: INFO: konnectivity-agent-s27mp started at 2022-11-24 04:58:37 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container konnectivity-agent ready: false, restart count 9 Nov 24 05:37:39.680: INFO: netserver-2 started at 2022-11-24 05:29:02 +0000 UTC (0+1 container statuses recorded) Nov 24 05:37:39.680: INFO: Container webserver ready: true, restart count 2 Nov 24 05:37:40.445: INFO: Latency metrics for node bootstrap-e2e-minion-group-qvgt [DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] tear down framework | framework.go:193 STEP: Destroying namespace "crd-publish-openapi-5487" for this suite. 11/24/22 05:37:40.445
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourcePublishOpenAPI\s\[Privileged\:ClusterAdmin\]\sworks\sfor\smultiple\sCRDs\sof\ssame\sgroup\sbut\sdifferent\sversions\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000f82960) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:31:50.618 Nov 24 05:31:50.618: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename crd-publish-openapi 11/24/22 05:31:50.62 Nov 24 05:31:50.659: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:31:52.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:31:54.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:31:56.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:31:58.698: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:00.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:02.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:04.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:06.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:08.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:10.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:12.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:14.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:16.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:18.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:20.699: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:20.738: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:32:20.738: INFO: Unexpected error: <*errors.errorString | 0xc00017fa50>: { s: "timed out waiting for the condition", } Nov 24 05:32:20.738: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000f82960) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:32:20.738: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:32:20.778 [DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceValidationRules\s\[Privileged\:ClusterAdmin\]\sMUST\sNOT\sfail\svalidation\sfor\screate\sof\sa\scustom\sresource\sthat\ssatisfies\sthe\sx\-kubernetes\-validations\srules$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0011baa50) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:10:44.798 Nov 24 05:10:44.798: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename crd-validation-expressions 11/24/22 05:10:44.8 Nov 24 05:10:44.839: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:46.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:48.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:50.880: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:52.880: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:54.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:56.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:58.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:00.880: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:02.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:04.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:06.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:08.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:10.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:12.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:14.879: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:14.918: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:14.918: INFO: Unexpected error: <*errors.errorString | 0xc0001ffa10>: { s: "timed out waiting for the condition", } Nov 24 05:11:14.918: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0011baa50) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:11:14.918: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:11:14.958 [DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceValidationRules\s\[Privileged\:ClusterAdmin\]\sMUST\sfail\svalidation\sfor\screate\sof\sa\scustom\sresource\sthat\sdoes\snot\ssatisfy\sthe\sx\-kubernetes\-validations\srules$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010fea50) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:10:43.029 Nov 24 05:10:43.029: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename crd-validation-expressions 11/24/22 05:10:43.031 Nov 24 05:10:43.070: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:45.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:47.109: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:49.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:51.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:53.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:55.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:57.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:10:59.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:01.109: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:03.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:05.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:07.109: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:09.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:11.110: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:13.112: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:13.151: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:11:13.151: INFO: Unexpected error: <*errors.errorString | 0xc000207d70>: { s: "timed out waiting for the condition", } Nov 24 05:11:13.151: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010fea50) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] test/e2e/framework/node/init/init.go:32 Nov 24 05:11:13.152: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:11:13.191 [DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sDiscovery\sshould\saccurately\sdetermine\spresent\sand\smissing\sresources$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0011fbd10) test/e2e/framework/framework.go:241 +0x96ffrom junit_01.xml
[BeforeEach] [sig-api-machinery] Discovery set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:18:40.438 Nov 24 05:18:40.438: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename discovery 11/24/22 05:18:40.439 Nov 24 05:18:40.479: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:42.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:44.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:46.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:48.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:50.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:52.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:54.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:56.518: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:18:58.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:00.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:02.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:04.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:06.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:08.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:10.519: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:10.558: INFO: Unexpected error while creating namespace: Post "https://34.83.136.178/api/v1/namespaces": dial tcp 34.83.136.178:443: connect: connection refused Nov 24 05:19:10.558: INFO: Unexpected error: <*errors.errorString | 0xc00011dd80>: { s: "timed out waiting for the condition", } Nov 24 05:19:10.558: FAIL: timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0011fbd10) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] Discovery test/e2e/framework/node/init/init.go:32 Nov 24 05:19:10.558: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] Discovery dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:19:10.598 [DeferCleanup (Each)] [sig-api-machinery] Discovery tear down framework | framework.go:193
Filter through log files | View test history on testgrid
go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sDiscovery\sshould\svalidate\sPreferredVersion\sfor\seach\sAPIGroup\s\[Conformance\]$'
test/e2e/framework/framework.go:241 k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001143d10) test/e2e/framework/framework.go:241 +0x96f
[BeforeEach] [sig-api-machinery] Discovery set up framework | framework.go:178 STEP: Creating a kubernetes client 11/24/22 05:09:27.401 Nov 24 05:09:27.402: INFO: >>> kubeConfig: /workspace/.kube/config STEP: Building a namespace api object, basename discovery 11/24/22 05:09:27.403 Nov 24 05:11:49.164: INFO: Unexpected error: <*fmt.wrapError | 0xc00124c000>: { msg: "wait for service account \"default\" in namespace \"discovery-5408\": timed out waiting for the condition", err: <*errors.errorString | 0xc000205cd0>{ s: "timed out waiting for the condition", }, } Nov 24 05:11:49.164: FAIL: wait for service account "default" in namespace "discovery-5408": timed out waiting for the condition Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc001143d10) test/e2e/framework/framework.go:241 +0x96f [AfterEach] [sig-api-machinery] Discovery test/e2e/framework/node/init/init.go:32 Nov 24 05:11:49.164: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [DeferCleanup (Each)] [sig-api-machinery] Discovery dump namespaces | framework.go:196 STEP: dump namespace information after failure 11/24/22 05:11:49.336 STEP: Collecting events from namespace "discovery-5408". 11/24/22 05:11:49.336 STEP: Found 0 events. 11/24/22 05:11:49.379 Nov 24 05:11:49.499: INFO: POD NODE PHASE GRACE CONDITIONS Nov 24 05:11:49.499: INFO: Nov 24 05:11:49.574: INFO: Logging node info for node bootstrap-e2e-master Nov 24 05:11:49.616: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master c97f96bb-2961-49ae-9f92-8c1d200f4cf4 7337 0 2022-11-24 04:58:19 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-24 04:58:19 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-24 04:58:37 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kube-controller-manager Update v1 2022-11-24 04:58:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}},"f:taints":{}}} } {kubelet Update v1 2022-11-24 05:09:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-4/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858374656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596230656 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-24 04:58:37 +0000 UTC,LastTransitionTime:2022-11-24 04:58:37 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-24 05:09:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-24 05:09:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-24 05:09:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:19 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-24 05:09:26 +0000 UTC,LastTransitionTime:2022-11-24 04:58:39 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:34.83.136.178,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-4.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:15c4ea8c07e55e5a1f5f85ebe28608da,SystemUUID:15c4ea8c-07e5-5e5a-1f5f-85ebe28608da,BootID:e8f95bbe-653b-4bed-a463-0f0a71432d6f,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.42+7530baa9a186a0,KubeProxyVersion:v1.27.0-alpha.0.42+7530baa9a186a0,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:124990265,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:b83c1d70989e1fe87583607bf5aee1ee34e52773d4755b95f5cf5a451962f3a4 registry.k8s.io/etcd:3.5.5-0],SizeBytes:102417044,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.42_7530baa9a186a0],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-s