ci-cri-containerd-e2e-cos-gce-proto #1595974545829793792 Results

Test Failures

Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] patching/updating a validating webhook should work [Conformance] 11s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\spatching\/updating\sa\svalidating\swebhook\sshould\swork\s\[Conformance\]$'

test/e2e/apimachinery/webhook.go:853
k8s.io/kubernetes/test/e2e/apimachinery.deployWebhookAndService(0xc00124ae10, {0xc001a30690, 0x2c}, 0xc002ee0b40, 0x20fb, 0x20fc)
	test/e2e/apimachinery/webhook.go:853 +0x13da
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.1()
	test/e2e/apimachinery/webhook.go:102 +0x226
There were additional failures detected after the initial failure:
[FAILED]
Nov 25 03:16:03.395: failed to list events in namespace "webhook-9528": Get "https://35.247.30.25/api/v1/namespaces/webhook-9528/events": dial tcp 35.247.30.25:443: connect: connection refused
In [DeferCleanup (Each)] at: test/e2e/framework/debug/dump.go:44
----------
[FAILED]
Nov 25 03:16:03.474: Couldn't delete ns: "webhook-9528": Delete "https://35.247.30.25/api/v1/namespaces/webhook-9528": dial tcp 35.247.30.25:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://35.247.30.25/api/v1/namespaces/webhook-9528", Err:(*net.OpError)(0xc0032fecd0)}),Couldn't delete ns: "webhook-9528-markers": Delete "https://35.247.30.25/api/v1/namespaces/webhook-9528-markers": dial tcp 35.247.30.25:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://35.247.30.25/api/v1/namespaces/webhook-9528-markers", Err:(*net.OpError)(0xc00237dc20)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370

				[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:15:52.142
Nov 25 03:15:52.142: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/25/22 03:15:52.144
STEP: Waiting for a default service account to be provisioned in namespace 11/25/22 03:15:52.411
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/25/22 03:15:52.507
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:90
STEP: Setting up server cert 11/25/22 03:15:52.756
STEP: Create role binding to let webhook read extension-apiserver-authentication 11/25/22 03:15:53.255
STEP: Deploying the webhook pod 11/25/22 03:15:53.336
STEP: Wait for the deployment to be ready 11/25/22 03:15:53.477
Nov 25 03:15:53.739: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:0, UpdatedReplicas:0, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"NewReplicaSetCreated", Message:"Created new replica set \"sample-webhook-deployment-865554f4d9\""}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:15:55.880: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:15:57.866: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:15:59.796: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 15, 53, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
STEP: Deploying the webhook service 11/25/22 03:16:01.925
STEP: Verifying the service has paired with the endpoint 11/25/22 03:16:02.118
Nov 25 03:16:03.119: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:16:03.158: INFO: Unexpected error: waiting for service webhook-9528/e2e-test-webhook have 1 endpoint: 
    <*url.Error | 0xc001902b70>: {
        Op: "Get",
        URL: "https://35.247.30.25/api/v1/namespaces/webhook-9528/endpoints",
        Err: <*net.OpError | 0xc00237d6d0>{
            Op: "dial",
            Net: "tcp",
            Source: nil,
            Addr: <*net.TCPAddr | 0xc0013fbad0>{
                IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 35, 247, 30, 25],
                Port: 443,
                Zone: "",
            },
            Err: <*os.SyscallError | 0xc001090f60>{
                Syscall: "connect",
                Err: <syscall.Errno>0x6f,
            },
        },
    }
Nov 25 03:16:03.158: FAIL: waiting for service webhook-9528/e2e-test-webhook have 1 endpoint: Get "https://35.247.30.25/api/v1/namespaces/webhook-9528/endpoints": dial tcp 35.247.30.25:443: connect: connection refused

Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.deployWebhookAndService(0xc00124ae10, {0xc001a30690, 0x2c}, 0xc002ee0b40, 0x20fb, 0x20fc)
	test/e2e/apimachinery/webhook.go:853 +0x13da
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.1()
	test/e2e/apimachinery/webhook.go:102 +0x226
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:16:03.158: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:16:03.355
STEP: Collecting events from namespace "webhook-9528". 11/25/22 03:16:03.355
Nov 25 03:16:03.394: INFO: Unexpected error: failed to list events in namespace "webhook-9528": 
    <*url.Error | 0xc001903020>: {
        Op: "Get",
        URL: "https://35.247.30.25/api/v1/namespaces/webhook-9528/events",
        Err: <*net.OpError | 0xc00237d900>{
            Op: "dial",
            Net: "tcp",
            Source: nil,
            Addr: <*net.TCPAddr | 0xc002131e30>{
                IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 35, 247, 30, 25],
                Port: 443,
                Zone: "",
            },
            Err: <*os.SyscallError | 0xc0010912c0>{
                Syscall: "connect",
                Err: <syscall.Errno>0x6f,
            },
        },
    }
Nov 25 03:16:03.395: FAIL: failed to list events in namespace "webhook-9528": Get "https://35.247.30.25/api/v1/namespaces/webhook-9528/events": dial tcp 35.247.30.25:443: connect: connection refused

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/debug.dumpEventsInNamespace(0xc00226e5c0, {0xc0009484c0, 0xc})
	test/e2e/framework/debug/dump.go:44 +0x191
k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc0013b29c0}, {0xc0009484c0, 0xc})
	test/e2e/framework/debug/dump.go:62 +0x8d
k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc00226e650?, {0xc0009484c0?, 0x7fa7740?})
	test/e2e/framework/debug/init/init.go:34 +0x32
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
	test/e2e/framework/framework.go:274 +0x6d
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc00124ae10)
	test/e2e/framework/framework.go:271 +0x179
reflect.Value.call({0x6627cc0?, 0xc001453c30?, 0xc000e7bfb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0xc0025f6088?})
	/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc001453c30?, 0x29449fc?}, {0xae73300?, 0xc000e7bf80?, 0x2a6d786?})
	/usr/local/go/src/reflect/value.go:368 +0xbc
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "webhook-9528" for this suite. 11/25/22 03:16:03.395
STEP: Destroying namespace "webhook-9528-markers" for this suite. 11/25/22 03:16:03.435
Nov 25 03:16:03.474: FAIL: Couldn't delete ns: "webhook-9528": Delete "https://35.247.30.25/api/v1/namespaces/webhook-9528": dial tcp 35.247.30.25:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://35.247.30.25/api/v1/namespaces/webhook-9528", Err:(*net.OpError)(0xc0032fecd0)}),Couldn't delete ns: "webhook-9528-markers": Delete "https://35.247.30.25/api/v1/namespaces/webhook-9528-markers": dial tcp 35.247.30.25:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://35.247.30.25/api/v1/namespaces/webhook-9528-markers", Err:(*net.OpError)(0xc00237dc20)})

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
	test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc00124ae10)
	test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc001453b80?, 0xc000e7afb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
	/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc001453b80?, 0x0?}, {0xae73300?, 0x5?, 0xc000949760?})
	/usr/local/go/src/reflect/value.go:368 +0xbc

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny attaching pod [Conformance] 2m37s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sbe\sable\sto\sdeny\sattaching\spod\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010ade00)
	test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260

runtime error: invalid memory address or nil pointer dereference

Full Stack Trace
  k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
  	test/e2e/apimachinery/webhook.go:1680 +0x46
  k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
  	test/e2e/apimachinery/webhook.go:106 +0x2f

				[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:16:05.192
Nov 25 03:16:05.192: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/25/22 03:16:05.194
Nov 25 03:16:05.233: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:07.274: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:09.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:11.274: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:13.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:15.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:17.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:19.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:21.274: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:23.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:25.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:27.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:29.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:31.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:33.273: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:18:37.967: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc000ede000>: {
        msg: "wait for service account \"default\" in namespace \"webhook-1928\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc000113cb0>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:18:37.967: FAIL: wait for service account "default" in namespace "webhook-1928": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010ade00)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:18:37.967: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:18:38.05
STEP: Collecting events from namespace "webhook-1928". 11/25/22 03:18:38.051
STEP: Found 0 events. 11/25/22 03:18:38.092
Nov 25 03:18:38.139: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:18:38.139: INFO: 
Nov 25 03:18:38.241: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:18:38.372: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 14600 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:08 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:18:38.372: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:18:38.507: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:18:39.087: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container kube-controller-manager ready: false, restart count 5
Nov 25 03:18:39.087: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container kube-scheduler ready: true, restart count 5
Nov 25 03:18:39.087: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container kube-addon-manager ready: true, restart count 4
Nov 25 03:18:39.087: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container l7-lb-controller ready: false, restart count 6
Nov 25 03:18:39.087: INFO: metadata-proxy-v0.1-vkhjm started at 2022-11-25 03:01:26 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:18:39.087: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:18:39.087: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container etcd-container ready: true, restart count 0
Nov 25 03:18:39.087: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container etcd-container ready: true, restart count 3
Nov 25 03:18:39.087: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container kube-apiserver ready: true, restart count 1
Nov 25 03:18:39.087: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.087: INFO: 	Container konnectivity-server-container ready: true, restart count 1
Nov 25 03:18:39.713: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 25 03:18:39.713: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:18:39.763: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 14713 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2480":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:11:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:14:15 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:18:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:15 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:14:15 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:14:15 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:14:15 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:14:15 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:15 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:15 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:18:39.764: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:18:39.811: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:18:39.989: INFO: metrics-server-v0.5.2-867b8754b9-mkqv8 started at 2022-11-25 03:01:55 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container metrics-server ready: false, restart count 6
Nov 25 03:18:39.989: INFO: 	Container metrics-server-nanny ready: false, restart count 7
Nov 25 03:18:39.989: INFO: test-deployment-tw5kw-54bc444df-5lzn2 started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container httpd ready: true, restart count 2
Nov 25 03:18:39.989: INFO: hostexec-bootstrap-e2e-minion-group-2842-khxh6 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 25 03:18:39.989: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:03:12 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container csi-attacher ready: false, restart count 6
Nov 25 03:18:39.989: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 25 03:18:39.989: INFO: 	Container csi-resizer ready: false, restart count 6
Nov 25 03:18:39.989: INFO: 	Container csi-snapshotter ready: false, restart count 6
Nov 25 03:18:39.989: INFO: 	Container hostpath ready: false, restart count 6
Nov 25 03:18:39.989: INFO: 	Container liveness-probe ready: false, restart count 6
Nov 25 03:18:39.989: INFO: 	Container node-driver-registrar ready: false, restart count 6
Nov 25 03:18:39.989: INFO: kube-proxy-bootstrap-e2e-minion-group-2842 started at 2022-11-25 03:01:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container kube-proxy ready: false, restart count 6
Nov 25 03:18:39.989: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:09:21 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container csi-attacher ready: true, restart count 4
Nov 25 03:18:39.989: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 25 03:18:39.989: INFO: 	Container csi-resizer ready: true, restart count 4
Nov 25 03:18:39.989: INFO: 	Container csi-snapshotter ready: true, restart count 4
Nov 25 03:18:39.989: INFO: 	Container hostpath ready: true, restart count 4
Nov 25 03:18:39.989: INFO: 	Container liveness-probe ready: true, restart count 4
Nov 25 03:18:39.989: INFO: 	Container node-driver-registrar ready: true, restart count 4
Nov 25 03:18:39.989: INFO: sample-webhook-deployment-865554f4d9-s2mvw started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container sample-webhook ready: true, restart count 3
Nov 25 03:18:39.989: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:16:00 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container csi-attacher ready: false, restart count 4
Nov 25 03:18:39.989: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:18:39.989: INFO: 	Container csi-resizer ready: false, restart count 4
Nov 25 03:18:39.989: INFO: 	Container csi-snapshotter ready: false, restart count 4
Nov 25 03:18:39.989: INFO: 	Container hostpath ready: false, restart count 4
Nov 25 03:18:39.989: INFO: 	Container liveness-probe ready: false, restart count 4
Nov 25 03:18:39.989: INFO: 	Container node-driver-registrar ready: false, restart count 4
Nov 25 03:18:39.989: INFO: konnectivity-agent-xr8pf started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container konnectivity-agent ready: false, restart count 6
Nov 25 03:18:39.989: INFO: pod-server-1 started at 2022-11-25 03:15:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:18:39.989: INFO: ss2-0 started at 2022-11-25 03:15:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:18:39.989: INFO: netserver-0 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container webserver ready: true, restart count 3
Nov 25 03:18:39.989: INFO: metadata-proxy-v0.1-4nskz started at 2022-11-25 03:01:32 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:18:39.989: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:18:39.989: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:18:40.850: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-2842
Nov 25 03:18:40.850: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:18:40.902: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 14714 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-1554":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:14:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:15:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:18:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:14:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:14:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:14:17 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:14:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-1494^7981ca50-6c6f-11ed-97be-7e6aeec89790,DevicePath:,},},Config:nil,},}
Nov 25 03:18:40.903: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:18:41.021: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:18:41.252: INFO: test-container-pod started at 2022-11-25 03:12:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container webserver ready: false, restart count 2
Nov 25 03:18:41.252: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:15:52 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container csi-attacher ready: false, restart count 1
Nov 25 03:18:41.252: INFO: 	Container csi-provisioner ready: false, restart count 1
Nov 25 03:18:41.252: INFO: 	Container csi-resizer ready: false, restart count 1
Nov 25 03:18:41.252: INFO: 	Container csi-snapshotter ready: false, restart count 1
Nov 25 03:18:41.252: INFO: 	Container hostpath ready: false, restart count 1
Nov 25 03:18:41.252: INFO: 	Container liveness-probe ready: false, restart count 1
Nov 25 03:18:41.252: INFO: 	Container node-driver-registrar ready: false, restart count 1
Nov 25 03:18:41.252: INFO: pod-subpath-test-dynamicpv-xn2k started at 2022-11-25 03:15:57 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container test-container-subpath-dynamicpv-xn2k ready: false, restart count 0
Nov 25 03:18:41.252: INFO: 	Container test-container-volume-dynamicpv-xn2k ready: false, restart count 0
Nov 25 03:18:41.252: INFO: csi-mockplugin-0 started at 2022-11-25 03:13:23 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:18:41.252: INFO: 	Container driver-registrar ready: true, restart count 1
Nov 25 03:18:41.252: INFO: 	Container mock ready: true, restart count 1
Nov 25 03:18:41.252: INFO: kube-proxy-bootstrap-e2e-minion-group-4j20 started at 2022-11-25 03:01:30 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container kube-proxy ready: false, restart count 6
Nov 25 03:18:41.252: INFO: dns-test-fa248686-6c83-425f-834a-b5c24a5cd609 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container jessie-querier ready: false, restart count 2
Nov 25 03:18:41.252: INFO: 	Container querier ready: false, restart count 2
Nov 25 03:18:41.252: INFO: 	Container webserver ready: false, restart count 2
Nov 25 03:18:41.252: INFO: execpodndnps started at 2022-11-25 03:15:48 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container agnhost-container ready: true, restart count 1
Nov 25 03:18:41.252: INFO: agnhost started at 2022-11-25 03:03:43 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container agnhost ready: false, restart count 0
Nov 25 03:18:41.252: INFO: netserver-1 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container webserver ready: false, restart count 1
Nov 25 03:18:41.252: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:09:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 25 03:18:41.252: INFO: host-test-container-pod started at 2022-11-25 03:12:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 25 03:18:41.252: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:13:23 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container csi-attacher ready: true, restart count 1
Nov 25 03:18:41.252: INFO: pvc-volume-tester-5snsv started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:18:41.252: INFO: ss2-2 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container webserver ready: true, restart count 1
Nov 25 03:18:41.252: INFO: pod2 started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 25 03:18:41.252: INFO: csi-mockplugin-0 started at 2022-11-25 03:09:45 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 25 03:18:41.252: INFO: 	Container driver-registrar ready: true, restart count 4
Nov 25 03:18:41.252: INFO: 	Container mock ready: true, restart count 4
Nov 25 03:18:41.252: INFO: konnectivity-agent-wbnhz started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container konnectivity-agent ready: false, restart count 7
Nov 25 03:18:41.252: INFO: metadata-proxy-v0.1-s4hwh started at 2022-11-25 03:01:31 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:18:41.252: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:18:41.252: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:18:41.840: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-4j20
Nov 25 03:18:41.840: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:18:41.881: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 14639 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:14:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:14:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:14:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:14:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:14:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:14:21 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:18:41.882: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:18:41.927: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:18:42.170: INFO: pod1 started at 2022-11-25 03:13:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:18:42.170: INFO: metadata-proxy-v0.1-pcvjx started at 2022-11-25 03:01:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:18:42.170: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:18:42.170: INFO: coredns-6d97d5ddb-ts7gx started at 2022-11-25 03:01:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container coredns ready: false, restart count 7
Nov 25 03:18:42.170: INFO: coredns-6d97d5ddb-r9926 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container coredns ready: false, restart count 7
Nov 25 03:18:42.170: INFO: ss2-1 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container webserver ready: true, restart count 0
Nov 25 03:18:42.170: INFO: nfs-server started at 2022-11-25 03:09:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container nfs-server ready: true, restart count 6
Nov 25 03:18:42.170: INFO: recycler-for-nfs-h8mdt started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container pv-recycler ready: false, restart count 0
Nov 25 03:18:42.170: INFO: volume-snapshot-controller-0 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container volume-snapshot-controller ready: false, restart count 6
Nov 25 03:18:42.170: INFO: konnectivity-agent-gd8tk started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container konnectivity-agent ready: true, restart count 5
Nov 25 03:18:42.170: INFO: pvc-volume-tester-s584h started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:18:42.170: INFO: kube-proxy-bootstrap-e2e-minion-group-q7zm started at 2022-11-25 03:01:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container kube-proxy ready: false, restart count 7
Nov 25 03:18:42.170: INFO: csi-mockplugin-0 started at 2022-11-25 03:11:56 +0000 UTC (0+4 container statuses recorded)
Nov 25 03:18:42.170: INFO: 	Container busybox ready: true, restart count 4
Nov 25 03:18:42.170: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:18:42.170: INFO: 	Container driver-registrar ready: false, restart count 3
Nov 25 03:18:42.170: INFO: 	Container mock ready: false, restart count 3
Nov 25 03:18:42.171: INFO: pod-projected-secrets-fd485c27-166a-49eb-b3cc-a98f3b0470fa started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container projected-secret-volume-test ready: false, restart count 0
Nov 25 03:18:42.171: INFO: l7-default-backend-8549d69d99-k696w started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container default-http-backend ready: true, restart count 0
Nov 25 03:18:42.171: INFO: execpodlc59j started at 2022-11-25 03:13:12 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container agnhost-container ready: false, restart count 3
Nov 25 03:18:42.171: INFO: netserver-2 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:18:42.171: INFO: downwardapi-volume-927901cf-394f-4333-8c62-7d60bf4ee648 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:18:42.171: INFO: kube-dns-autoscaler-5f6455f985-77mxq started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container autoscaler ready: false, restart count 7
Nov 25 03:18:42.171: INFO: pod-secrets-0a07040a-2718-4694-83b9-20611c00d694 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 25 03:18:42.171: INFO: 	Container dels-volume-test ready: false, restart count 0
Nov 25 03:18:42.171: INFO: 	Container upds-volume-test ready: false, restart count 0
Nov 25 03:18:42.171: INFO: pod-handle-http-request started at 2022-11-25 03:15:49 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:18:42.171: INFO: 	Container container-handle-http-request ready: true, restart count 1
Nov 25 03:18:42.171: INFO: 	Container container-handle-https-request ready: true, restart count 1
Nov 25 03:18:42.479: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-q7zm
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "webhook-1928" for this suite. 11/25/22 03:18:42.479

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny pod and configmap creation [Conformance] 2m1s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sbe\sable\sto\sdeny\spod\sand\sconfigmap\screation\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0004ede00)
	test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260

runtime error: invalid memory address or nil pointer dereference

Full Stack Trace
  k8s.io/kubernetes/test/e2e/apimachinery.cleanWebhookTest({0x0, 0x0}, {0x0, 0x0})
  	test/e2e/apimachinery/webhook.go:1680 +0x46
  k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.2()
  	test/e2e/apimachinery/webhook.go:106 +0x2f

				[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:36:50.612
Nov 25 03:36:50.612: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/25/22 03:36:50.614
Nov 25 03:38:50.659: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc000dd8020>: {
        msg: "wait for service account \"default\" in namespace \"webhook-7870\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc0001f7a10>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:38:50.659: FAIL: wait for service account "default" in namespace "webhook-7870": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0004ede00)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:38:50.659: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:38:50.701
STEP: Collecting events from namespace "webhook-7870". 11/25/22 03:38:50.701
STEP: Found 0 events. 11/25/22 03:38:50.741
Nov 25 03:38:50.783: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:38:50.783: INFO: 
Nov 25 03:38:50.873: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:38:50.916: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 20981 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:37:34 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:38:50.916: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:38:50.973: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:38:51.045: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:38:51.045: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:38:51.091: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 20919 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3961":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:33:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:34:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:37:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3961^e6de85f2-6c71-11ed-9fff-d6801b68d09f,DevicePath:,},},Config:nil,},}
Nov 25 03:38:51.092: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:38:51.171: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:38:51.284: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:38:51.284: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:38:51.329: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 20914 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-5097":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-2265":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:26:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:34:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:59 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:59 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:59 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:36:59 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:38:51.330: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:38:51.440: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:38:51.561: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:38:51.561: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:38:51.607: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 20729 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:34:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:38:51.607: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:38:51.671: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:38:51.735: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "webhook-7870" for this suite. 11/25/22 03:38:51.736

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should deny crd creation [Conformance] 1m24s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\sdeny\scrd\screation\s\[Conformance\]$'

test/e2e/apimachinery/webhook.go:2041
k8s.io/kubernetes/test/e2e/apimachinery.registerValidatingWebhookForCRD(0xc001151e00, {0xc0012c22f0, 0xc}, 0xc00197c820, 0x0?)
	test/e2e/apimachinery/webhook.go:2041 +0x792
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.12()
	test/e2e/apimachinery/webhook.go:309 +0x45
				[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:05:19.75
Nov 25 03:05:19.750: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/25/22 03:05:19.751
STEP: Waiting for a default service account to be provisioned in namespace 11/25/22 03:05:19.906
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/25/22 03:05:20.001
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:90
STEP: Setting up server cert 11/25/22 03:05:20.236
STEP: Create role binding to let webhook read extension-apiserver-authentication 11/25/22 03:05:20.943
Nov 25 03:05:21.065: INFO: role binding webhook-auth-reader already exists
STEP: Deploying the webhook pod 11/25/22 03:05:21.065
STEP: Wait for the deployment to be ready 11/25/22 03:05:21.342
Nov 25 03:05:21.557: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:0, UpdatedReplicas:0, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"NewReplicaSetCreated", Message:"Created new replica set \"sample-webhook-deployment-865554f4d9\""}, v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:23.643: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:25.619: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:27.616: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:29.652: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:31.633: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:33.644: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:35.620: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:37.610: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:39.607: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:41.606: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:43.607: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:45.625: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:47.650: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:49.627: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:05:51.606: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 5, 21, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
STEP: Deploying the webhook service 11/25/22 03:05:53.625
STEP: Verifying the service has paired with the endpoint 11/25/22 03:05:53.73
Nov 25 03:05:54.731: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
[It] should deny crd creation [Conformance]
  test/e2e/apimachinery/webhook.go:308
STEP: Registering the crd webhook via the AdmissionRegistration API 11/25/22 03:05:54.78
Nov 25 03:06:04.984: INFO: Waiting for webhook configuration to be ready...
Nov 25 03:06:15.221: INFO: Waiting for webhook configuration to be ready...
Nov 25 03:06:25.440: INFO: Waiting for webhook configuration to be ready...
Nov 25 03:06:35.644: INFO: Waiting for webhook configuration to be ready...
Nov 25 03:06:41.328: INFO: Waiting for webhook configuration to be ready...
Nov 25 03:06:41.328: INFO: Unexpected error: waiting for webhook configuration to be ready: 
    <*errors.errorString | 0xc000115d90>: {
        s: "timed out waiting for the condition",
    }
Nov 25 03:06:41.328: FAIL: waiting for webhook configuration to be ready: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.registerValidatingWebhookForCRD(0xc001151e00, {0xc0012c22f0, 0xc}, 0xc00197c820, 0x0?)
	test/e2e/apimachinery/webhook.go:2041 +0x792
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.12()
	test/e2e/apimachinery/webhook.go:309 +0x45
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:06:41.328: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:06:41.799
STEP: Collecting events from namespace "webhook-4343". 11/25/22 03:06:41.799
STEP: Found 11 events. 11/25/22 03:06:41.862
Nov 25 03:06:41.862: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: { } Scheduled: Successfully assigned webhook-4343/sample-webhook-deployment-865554f4d9-rljbn to bootstrap-e2e-minion-group-4j20
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:21 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-865554f4d9 to 1
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:21 +0000 UTC - event for sample-webhook-deployment-865554f4d9: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-865554f4d9-rljbn
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:22 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:22 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} Created: Created container sample-webhook
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:23 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} Started: Started container sample-webhook
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:25 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} Killing: Stopping container sample-webhook
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:26 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:27 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} Unhealthy: Readiness probe failed: Get "https://10.64.2.87:8444/readyz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:30 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} Unhealthy: Readiness probe failed: Get "https://10.64.2.90:8444/readyz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Nov 25 03:06:41.862: INFO: At 2022-11-25 03:05:31 +0000 UTC - event for sample-webhook-deployment-865554f4d9-rljbn: {kubelet bootstrap-e2e-minion-group-4j20} BackOff: Back-off restarting failed container sample-webhook in pod sample-webhook-deployment-865554f4d9-rljbn_webhook-4343(770097cf-8a28-4bc6-9878-ab40d6cd3cc2)
Nov 25 03:06:41.919: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:06:41.919: INFO: 
Nov 25 03:06:41.982: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:06:42.038: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 583 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:01:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:06:42.038: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:06:42.104: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:06:42.174: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:06:42.174: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:06:42.221: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 7205 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3405":"bootstrap-e2e-minion-group-2842","csi-hostpath-volume-expand-5942":"bootstrap-e2e-minion-group-2842","csi-mock-csi-mock-volumes-2994":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:04:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:04:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:05:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-5942^bbbc9151-6c6d-11ed-99ff-ae8414269b99],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-5942^bbbc9151-6c6d-11ed-99ff-ae8414269b99,DevicePath:,},},Config:nil,},}
Nov 25 03:06:42.222: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:06:42.287: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:06:42.375: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:06:42.375: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:06:42.427: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 7540 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-5327":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:01:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:04:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:06:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:06:42.427: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:06:42.490: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:06:42.573: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:06:42.573: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:06:42.651: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 5914 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:04:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:04:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:06:42.652: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:06:42.734: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:06:42.877: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
STEP: Collecting events from namespace "webhook-4343-markers". 11/25/22 03:06:42.877
STEP: Found 0 events. 11/25/22 03:06:42.959
Nov 25 03:06:43.051: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:06:43.051: INFO: 
Nov 25 03:06:43.180: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:06:43.253: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 583 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:01:46 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:01:46 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:06:43.254: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:06:43.326: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:06:43.414: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:06:43.414: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:06:43.475: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 7205 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-3405":"bootstrap-e2e-minion-group-2842","csi-hostpath-volume-expand-5942":"bootstrap-e2e-minion-group-2842","csi-mock-csi-mock-volumes-2994":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:04:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:04:26 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:05:50 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:04:26 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-expand-5942^bbbc9151-6c6d-11ed-99ff-ae8414269b99],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-volume-expand-5942^bbbc9151-6c6d-11ed-99ff-ae8414269b99,DevicePath:,},},Config:nil,},}
Nov 25 03:06:43.475: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:06:43.544: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:06:43.667: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:06:43.667: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:06:43.733: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 7540 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-5327":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:01:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:04:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:06:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:06:07 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:06:43.734: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:06:43.930: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:06:44.081: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:06:44.081: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:06:44.196: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 5914 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:04:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:04:57 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:04:20 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:04:57 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:06:44.196: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:06:44.309: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:06:44.508: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "webhook-4343" for this suite. 11/25/22 03:06:44.508
STEP: Destroying namespace "webhook-4343-markers" for this suite. 11/25/22 03:06:44.598

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should honor timeout [Conformance] 45s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAdmissionWebhook\s\[Privileged\:ClusterAdmin\]\sshould\shonor\stimeout\s\[Conformance\]$'

test/e2e/apimachinery/webhook.go:853
k8s.io/kubernetes/test/e2e/apimachinery.deployWebhookAndService(0xc000cf9e00, {0xc004f6c540, 0x2c}, 0xc0050201e0, 0x20fb, 0x20fc)
	test/e2e/apimachinery/webhook.go:853 +0x13da
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.1()
	test/e2e/apimachinery/webhook.go:102 +0x226
				[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:41:29.457
Nov 25 03:41:29.457: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename webhook 11/25/22 03:41:29.459
STEP: Waiting for a default service account to be provisioned in namespace 11/25/22 03:41:29.614
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/25/22 03:41:29.712
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:31
[BeforeEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:90
STEP: Setting up server cert 11/25/22 03:41:29.985
STEP: Create role binding to let webhook read extension-apiserver-authentication 11/25/22 03:41:30.498
Nov 25 03:41:30.565: INFO: role binding webhook-auth-reader already exists
STEP: Deploying the webhook pod 11/25/22 03:41:30.565
STEP: Wait for the deployment to be ready 11/25/22 03:41:30.675
Nov 25 03:41:30.972: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:41:33.112: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:41:35.107: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
Nov 25 03:41:37.079: INFO: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), LastTransitionTime:time.Date(2022, time.November, 25, 3, 41, 30, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"sample-webhook-deployment-865554f4d9\" is progressing."}}, CollisionCount:(*int32)(nil)}
STEP: Deploying the webhook service 11/25/22 03:41:39.302
STEP: Verifying the service has paired with the endpoint 11/25/22 03:41:39.669
Nov 25 03:41:40.670: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:41:41.670: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:41:42.669: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:41:43.670: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:41:44.669: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:41:45.670: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:42:11.033: INFO: Waiting for amount of service:e2e-test-webhook endpoints to be 1
Nov 25 03:42:11.075: INFO: Unexpected error: waiting for service webhook-6355/e2e-test-webhook have 1 endpoint: 
    <*errors.errorString | 0xc0001ffa10>: {
        s: "timed out waiting for the condition",
    }
Nov 25 03:42:11.075: FAIL: waiting for service webhook-6355/e2e-test-webhook have 1 endpoint: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.deployWebhookAndService(0xc000cf9e00, {0xc004f6c540, 0x2c}, 0xc0050201e0, 0x20fb, 0x20fc)
	test/e2e/apimachinery/webhook.go:853 +0x13da
k8s.io/kubernetes/test/e2e/apimachinery.glob..func28.1()
	test/e2e/apimachinery/webhook.go:102 +0x226
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:42:11.075: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[AfterEach] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/apimachinery/webhook.go:105
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:42:13.012
STEP: Collecting events from namespace "webhook-6355". 11/25/22 03:42:13.012
STEP: Found 13 events. 11/25/22 03:42:13.054
Nov 25 03:42:13.054: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: { } Scheduled: Successfully assigned webhook-6355/sample-webhook-deployment-865554f4d9-jcmkr to bootstrap-e2e-minion-group-4j20
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:30 +0000 UTC - event for sample-webhook-deployment: {deployment-controller } ScalingReplicaSet: Scaled up replica set sample-webhook-deployment-865554f4d9 to 1
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:30 +0000 UTC - event for sample-webhook-deployment-865554f4d9: {replicaset-controller } SuccessfulCreate: Created pod: sample-webhook-deployment-865554f4d9-jcmkr
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:32 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Pulled: Container image "registry.k8s.io/e2e-test-images/agnhost:2.43" already present on machine
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:32 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Created: Created container sample-webhook
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:32 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Started: Started container sample-webhook
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:33 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Killing: Stopping container sample-webhook
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:33 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Unhealthy: Readiness probe failed: Get "https://10.64.2.70:8444/readyz": EOF
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:34 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Unhealthy: Readiness probe failed: Get "https://10.64.2.70:8444/readyz": dial tcp 10.64.2.70:8444: connect: connection refused
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:34 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} SandboxChanged: Pod sandbox changed, it will be killed and re-created.
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:39 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Unhealthy: Readiness probe failed: Get "https://10.64.2.72:8444/readyz": dial tcp 10.64.2.72:8444: connect: connection refused
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:40 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} BackOff: Back-off restarting failed container sample-webhook in pod sample-webhook-deployment-865554f4d9-jcmkr_webhook-6355(9606d168-1ee9-45cf-84fc-b5c99dd483cb)
Nov 25 03:42:13.054: INFO: At 2022-11-25 03:41:56 +0000 UTC - event for sample-webhook-deployment-865554f4d9-jcmkr: {kubelet bootstrap-e2e-minion-group-4j20} Unhealthy: Readiness probe failed: Get "https://10.64.2.74:8444/readyz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Nov 25 03:42:13.095: INFO: POD                                         NODE                             PHASE    GRACE  CONDITIONS
Nov 25 03:42:13.095: INFO: sample-webhook-deployment-865554f4d9-jcmkr  bootstrap-e2e-minion-group-4j20  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2022-11-25 03:41:30 +0000 UTC  } {Ready False 0001-01-01 00:00:00 +0000 UTC 2022-11-25 03:41:55 +0000 UTC ContainersNotReady containers with unready status: [sample-webhook]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2022-11-25 03:41:55 +0000 UTC ContainersNotReady containers with unready status: [sample-webhook]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2022-11-25 03:41:30 +0000 UTC  }]
Nov 25 03:42:13.095: INFO: 
Nov 25 03:42:13.147: INFO: Unable to fetch webhook-6355/sample-webhook-deployment-865554f4d9-jcmkr/sample-webhook logs: an error on the server ("unknown") has prevented the request from succeeding (get pods sample-webhook-deployment-865554f4d9-jcmkr)
Nov 25 03:42:13.190: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:42:13.231: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 20981 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:37:34 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:42:13.231: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:42:13.277: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:42:13.320: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:42:13.320: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:42:13.365: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 24327 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8195":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-4097":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-6534":"bootstrap-e2e-minion-group-2842","csi-hostpath-volume-expand-5942":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:39:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:41:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:42:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8195^f46a6b65-6c72-11ed-ac0e-6ad658b1cfa8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},},Config:nil,},}
Nov 25 03:42:13.366: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:42:13.495: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:42:13.625: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:42:13.625: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:42:13.670: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 24328 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-5097":"bootstrap-e2e-minion-group-4j20","csi-hostpath-volume-expand-9814":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-1554":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-2265":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:26:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:39:40 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:42:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:42:13.671: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:42:13.715: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:42:13.757: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:42:13.757: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:42:13.799: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 23721 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:39:45 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:41:10 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:42:13.800: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:42:13.843: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:42:13.886: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
STEP: Collecting events from namespace "webhook-6355-markers". 11/25/22 03:42:13.886
STEP: Found 0 events. 11/25/22 03:42:13.931
Nov 25 03:42:13.972: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:42:13.972: INFO: 
Nov 25 03:42:14.014: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:42:14.059: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 20981 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:37:34 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:37:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:42:14.060: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:42:14.103: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:42:14.146: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:42:14.146: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:42:14.188: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 24400 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8195":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-4097":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-6534":"bootstrap-e2e-minion-group-2842","csi-hostpath-volume-expand-5942":"bootstrap-e2e-minion-group-2842","csi-mock-csi-mock-volumes-3961":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:39:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:41:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:42:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8195^f46a6b65-6c72-11ed-ac0e-6ad658b1cfa8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},},Config:nil,},}
Nov 25 03:42:14.188: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:42:14.235: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:42:14.280: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:42:14.280: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:42:14.322: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 24328 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-5097":"bootstrap-e2e-minion-group-4j20","csi-hostpath-volume-expand-9814":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-1554":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-2265":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:26:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:39:40 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:42:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:39:40 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:42:14.322: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:42:14.368: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:42:14.414: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:42:14.414: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:42:14.456: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 23721 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:39:45 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:41:10 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:39:45 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:41:10 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:42:14.456: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:42:14.552: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:42:14.627: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "webhook-6355" for this suite. 11/25/22 03:42:14.627
STEP: Destroying namespace "webhook-6355-markers" for this suite. 11/25/22 03:42:14.683

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] Aggregator Should be able to support the 1.17 Sample API Server using the current Aggregator [Conformance] 2m1s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sAggregator\sShould\sbe\sable\sto\ssupport\sthe\s1\.17\sSample\sAPI\sServer\susing\sthe\scurrent\sAggregator\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000cf84b0)
	test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[PANICKED]
Test Panicked
In [AfterEach] at: /usr/local/go/src/runtime/panic.go:260

runtime error: invalid memory address or nil pointer dereference

Full Stack Trace
  k8s.io/kubernetes/test/e2e/apimachinery.cleanTest({0x0, 0x0}, 0xc000b3ff80?, {0x0, 0x0})
  	test/e2e/apimachinery/aggregator.go:108 +0x46
  k8s.io/kubernetes/test/e2e/apimachinery.glob..func1.1()
  	test/e2e/apimachinery/aggregator.go:69 +0x36

				[BeforeEach] [sig-api-machinery] Aggregator
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:20:28.968
Nov 25 03:20:28.969: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename aggregator 11/25/22 03:20:28.97
Nov 25 03:22:29.331: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc0004a2020>: {
        msg: "wait for service account \"default\" in namespace \"aggregator-4967\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc0001ffa10>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:22:29.331: FAIL: wait for service account "default" in namespace "aggregator-4967": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000cf84b0)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] Aggregator
  test/e2e/apimachinery/aggregator.go:68
[AfterEach] [sig-api-machinery] Aggregator
  test/e2e/framework/node/init/init.go:32
Nov 25 03:22:29.331: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] Aggregator
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:22:29.374
STEP: Collecting events from namespace "aggregator-4967". 11/25/22 03:22:29.374
STEP: Found 0 events. 11/25/22 03:22:29.414
Nov 25 03:22:29.455: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:22:29.455: INFO: 
Nov 25 03:22:29.497: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:22:29.539: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 15645 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:22:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:22:29.539: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:22:29.586: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:22:29.630: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:22:29.630: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:22:29.672: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 15619 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2480":"bootstrap-e2e-minion-group-2842","csi-hostpath-volume-9455":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:11:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:19:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:22:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-9455^bb2344cc-6c6f-11ed-b8ba-ae517b0b92db],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:22:29.672: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:22:29.716: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:22:29.758: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:22:29.758: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:22:29.800: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 15660 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1494":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:19:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:19:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:22:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:22:29.800: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:22:29.844: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:22:29.886: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:22:29.886: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:22:29.929: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 14849 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-11-25 03:19:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:22:29.930: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:22:29.973: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:22:30.018: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-api-machinery] Aggregator
  tear down framework | framework.go:193
STEP: Destroying namespace "aggregator-4967" for this suite. 11/25/22 03:22:30.018

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] custom resource defaulting for requests and from storage works [Conformance] 30s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sresources\s\[Privileged\:ClusterAdmin\]\scustom\sresource\sdefaulting\sfor\srequests\sand\sfrom\sstorage\sworks\s\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000c80c30)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:16:03.035
Nov 25 03:16:03.035: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename custom-resource-definition 11/25/22 03:16:03.04
Nov 25 03:16:03.081: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:05.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:07.120: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:09.120: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:11.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:13.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:15.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:17.120: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:19.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:21.120: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:23.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:25.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:27.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:29.120: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:31.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:33.121: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:33.160: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:16:33.160: INFO: Unexpected error: 
    <*errors.errorString | 0xc0002498c0>: {
        s: "timed out waiting for the condition",
    }
Nov 25 03:16:33.160: FAIL: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000c80c30)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:16:33.161: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:16:33.2
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  tear down framework | framework.go:193

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin] should include custom resource definition resources in discovery documents [Conformance] 2m29s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceDefinition\sresources\s\[Privileged\:ClusterAdmin\]\sshould\sinclude\scustom\sresource\sdefinition\sresources\sin\sdiscovery\sdocuments\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0009e8c30)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:34:16.164
Nov 25 03:34:16.164: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename custom-resource-definition 11/25/22 03:34:16.166
Nov 25 03:34:16.206: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:18.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:20.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:22.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:24.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:26.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:28.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:30.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:32.246: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:36:36.626: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc0012e8060>: {
        msg: "wait for service account \"default\" in namespace \"custom-resource-definition-7673\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc000243a20>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:36:36.627: FAIL: wait for service account "default" in namespace "custom-resource-definition-7673": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0009e8c30)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:36:36.627: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:36:36.707
STEP: Collecting events from namespace "custom-resource-definition-7673". 11/25/22 03:36:36.707
STEP: Found 0 events. 11/25/22 03:36:36.748
Nov 25 03:36:36.788: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:36:36.788: INFO: 
Nov 25 03:36:36.833: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:36:36.878: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 19366 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:32:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:36:36.878: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:36:36.927: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:36:37.026: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container l7-lb-controller ready: true, restart count 10
Nov 25 03:36:37.026: INFO: metadata-proxy-v0.1-vkhjm started at 2022-11-25 03:01:26 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:37.026: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:37.026: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container etcd-container ready: true, restart count 0
Nov 25 03:36:37.026: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container etcd-container ready: true, restart count 4
Nov 25 03:36:37.026: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container kube-apiserver ready: true, restart count 2
Nov 25 03:36:37.026: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container kube-controller-manager ready: false, restart count 8
Nov 25 03:36:37.026: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container kube-scheduler ready: false, restart count 7
Nov 25 03:36:37.026: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container kube-addon-manager ready: true, restart count 5
Nov 25 03:36:37.026: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.026: INFO: 	Container konnectivity-server-container ready: true, restart count 1
Nov 25 03:36:37.507: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 25 03:36:37.507: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:36:37.609: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 20744 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3961":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:33:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:34:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3961^e6de85f2-6c71-11ed-9fff-d6801b68d09f,DevicePath:,},},Config:nil,},}
Nov 25 03:36:37.609: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:36:37.853: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:36:38.407: INFO: kube-proxy-bootstrap-e2e-minion-group-2842 started at 2022-11-25 03:01:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container kube-proxy ready: false, restart count 9
Nov 25 03:36:38.407: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:09:21 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container csi-attacher ready: false, restart count 6
Nov 25 03:36:38.407: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 25 03:36:38.407: INFO: 	Container csi-resizer ready: false, restart count 6
Nov 25 03:36:38.407: INFO: 	Container csi-snapshotter ready: false, restart count 6
Nov 25 03:36:38.407: INFO: 	Container hostpath ready: false, restart count 6
Nov 25 03:36:38.407: INFO: 	Container liveness-probe ready: false, restart count 6
Nov 25 03:36:38.407: INFO: 	Container node-driver-registrar ready: false, restart count 6
Nov 25 03:36:38.407: INFO: sample-webhook-deployment-865554f4d9-s2mvw started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container sample-webhook ready: false, restart count 6
Nov 25 03:36:38.407: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:16:00 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container csi-attacher ready: false, restart count 8
Nov 25 03:36:38.407: INFO: 	Container csi-provisioner ready: false, restart count 8
Nov 25 03:36:38.407: INFO: 	Container csi-resizer ready: false, restart count 8
Nov 25 03:36:38.407: INFO: 	Container csi-snapshotter ready: false, restart count 8
Nov 25 03:36:38.407: INFO: 	Container hostpath ready: false, restart count 8
Nov 25 03:36:38.407: INFO: 	Container liveness-probe ready: false, restart count 8
Nov 25 03:36:38.407: INFO: 	Container node-driver-registrar ready: false, restart count 8
Nov 25 03:36:38.407: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:33:38 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container csi-attacher ready: false, restart count 4
Nov 25 03:36:38.407: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:36:38.407: INFO: 	Container csi-resizer ready: false, restart count 4
Nov 25 03:36:38.407: INFO: 	Container csi-snapshotter ready: false, restart count 4
Nov 25 03:36:38.407: INFO: 	Container hostpath ready: false, restart count 4
Nov 25 03:36:38.407: INFO: 	Container liveness-probe ready: false, restart count 4
Nov 25 03:36:38.407: INFO: 	Container node-driver-registrar ready: false, restart count 4
Nov 25 03:36:38.407: INFO: agnhost-primary-75659d4b45-ppxww started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container primary ready: false, restart count 5
Nov 25 03:36:38.407: INFO: konnectivity-agent-xr8pf started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container konnectivity-agent ready: true, restart count 10
Nov 25 03:36:38.407: INFO: ss2-0 started at 2022-11-25 03:15:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container webserver ready: false, restart count 7
Nov 25 03:36:38.407: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 25 03:36:38.407: INFO: hostexec-bootstrap-e2e-minion-group-2842-8q8zg started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container agnhost-container ready: true, restart count 1
Nov 25 03:36:38.407: INFO: pod-ready started at 2022-11-25 03:25:48 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container pod-readiness-gate ready: true, restart count 4
Nov 25 03:36:38.407: INFO: metadata-proxy-v0.1-4nskz started at 2022-11-25 03:01:32 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:38.407: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:38.407: INFO: simpletest.rc-hp82q started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container nginx ready: true, restart count 0
Nov 25 03:36:38.407: INFO: oidc-discovery-validator started at 2022-11-25 03:25:48 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container oidc-discovery-validator ready: false, restart count 0
Nov 25 03:36:38.407: INFO: service-headless-wfqcn started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container service-headless ready: true, restart count 3
Nov 25 03:36:38.407: INFO: metrics-server-v0.5.2-867b8754b9-mkqv8 started at 2022-11-25 03:01:55 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container metrics-server ready: false, restart count 9
Nov 25 03:36:38.407: INFO: 	Container metrics-server-nanny ready: false, restart count 10
Nov 25 03:36:38.407: INFO: agnhost-replica-74c659fd6f-hkm8d started at 2022-11-25 03:25:52 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container replica ready: false, restart count 6
Nov 25 03:36:38.407: INFO: hostexec-bootstrap-e2e-minion-group-2842-6lzrc started at 2022-11-25 03:33:35 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container agnhost-container ready: false, restart count 4
Nov 25 03:36:38.407: INFO: hostexec-bootstrap-e2e-minion-group-2842-khxh6 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container agnhost-container ready: false, restart count 6
Nov 25 03:36:38.407: INFO: netserver-0 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:36:38.407: INFO: boom-server started at 2022-11-25 03:33:27 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container boom-server ready: false, restart count 0
Nov 25 03:36:38.407: INFO: csi-mockplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:36:38.407: INFO: 	Container driver-registrar ready: true, restart count 1
Nov 25 03:36:38.407: INFO: 	Container mock ready: true, restart count 1
Nov 25 03:36:38.407: INFO: frontend-78cd9dfff5-nj8mr started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container guestbook-frontend ready: false, restart count 6
Nov 25 03:36:38.407: INFO: pvc-volume-tester-w5szh started at 2022-11-25 03:33:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:38.407: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:03:12 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:38.407: INFO: 	Container csi-attacher ready: false, restart count 10
Nov 25 03:36:38.407: INFO: 	Container csi-provisioner ready: false, restart count 10
Nov 25 03:36:38.407: INFO: 	Container csi-resizer ready: false, restart count 10
Nov 25 03:36:38.407: INFO: 	Container csi-snapshotter ready: false, restart count 10
Nov 25 03:36:38.407: INFO: 	Container hostpath ready: false, restart count 10
Nov 25 03:36:38.407: INFO: 	Container liveness-probe ready: false, restart count 10
Nov 25 03:36:38.407: INFO: 	Container node-driver-registrar ready: false, restart count 10
Nov 25 03:36:44.774: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-2842
Nov 25 03:36:44.774: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:44.815: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 20813 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-2265":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:26:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:34:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:36:44.815: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:44.861: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:44.925: INFO: pvc-volume-tester-5snsv started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:44.925: INFO: ss2-2 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container webserver ready: false, restart count 6
Nov 25 03:36:44.925: INFO: pfpod started at 2022-11-25 03:33:28 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container portforwardtester ready: false, restart count 0
Nov 25 03:36:44.925: INFO: 	Container readiness ready: false, restart count 0
Nov 25 03:36:44.925: INFO: service-headless-6w72j started at 2022-11-25 03:33:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container service-headless ready: true, restart count 3
Nov 25 03:36:44.925: INFO: frontend-78cd9dfff5-8ttbp started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container guestbook-frontend ready: false, restart count 5
Nov 25 03:36:44.925: INFO: pod-handle-http-request started at 2022-11-25 03:33:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container container-handle-http-request ready: true, restart count 2
Nov 25 03:36:44.925: INFO: 	Container container-handle-https-request ready: true, restart count 2
Nov 25 03:36:44.925: INFO: csi-mockplugin-0 started at 2022-11-25 03:09:45 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container csi-provisioner ready: false, restart count 7
Nov 25 03:36:44.925: INFO: 	Container driver-registrar ready: false, restart count 7
Nov 25 03:36:44.925: INFO: 	Container mock ready: false, restart count 7
Nov 25 03:36:44.925: INFO: konnectivity-agent-wbnhz started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container konnectivity-agent ready: false, restart count 9
Nov 25 03:36:44.925: INFO: metadata-proxy-v0.1-s4hwh started at 2022-11-25 03:01:31 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:44.925: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:44.925: INFO: pod-e66b7f89-d8d9-46f2-90a6-f9d1fc4a6523 started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container test-container ready: false, restart count 0
Nov 25 03:36:44.925: INFO: pvc-volume-tester-hk7qj started at 2022-11-25 03:33:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:44.925: INFO: csi-mockplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:36:44.925: INFO: 	Container driver-registrar ready: true, restart count 1
Nov 25 03:36:44.925: INFO: 	Container mock ready: true, restart count 1
Nov 25 03:36:44.925: INFO: csi-mockplugin-0 started at 2022-11-25 03:13:23 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 25 03:36:44.925: INFO: 	Container driver-registrar ready: false, restart count 5
Nov 25 03:36:44.925: INFO: 	Container mock ready: false, restart count 5
Nov 25 03:36:44.925: INFO: kube-proxy-bootstrap-e2e-minion-group-4j20 started at 2022-11-25 03:01:30 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container kube-proxy ready: true, restart count 10
Nov 25 03:36:44.925: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container csi-attacher ready: false, restart count 1
Nov 25 03:36:44.925: INFO: 	Container csi-provisioner ready: false, restart count 1
Nov 25 03:36:44.925: INFO: 	Container csi-resizer ready: false, restart count 1
Nov 25 03:36:44.925: INFO: 	Container csi-snapshotter ready: false, restart count 1
Nov 25 03:36:44.925: INFO: 	Container hostpath ready: false, restart count 1
Nov 25 03:36:44.925: INFO: 	Container liveness-probe ready: false, restart count 1
Nov 25 03:36:44.925: INFO: 	Container node-driver-registrar ready: false, restart count 1
Nov 25 03:36:44.925: INFO: hostexec-bootstrap-e2e-minion-group-4j20-ffbwm started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:36:44.925: INFO: annotationupdateb256174a-aa90-4993-b6eb-102898f10910 started at 2022-11-25 03:33:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:36:44.925: INFO: simpletest.rc-89gpg started at 2022-11-25 03:33:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container nginx ready: true, restart count 1
Nov 25 03:36:44.925: INFO: netserver-1 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container webserver ready: false, restart count 4
Nov 25 03:36:44.925: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:09:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container csi-attacher ready: false, restart count 5
Nov 25 03:36:44.925: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:13:23 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.925: INFO: 	Container csi-attacher ready: true, restart count 5
Nov 25 03:36:45.507: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:45.507: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:36:45.549: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 20729 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:34:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:36:45.550: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:36:45.594: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:36:45.690: INFO: cronjob-test-27822453-w744f started at 2022-11-25 03:33:17 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container test ready: false, restart count 0
Nov 25 03:36:45.690: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:33:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container csi-attacher ready: false, restart count 2
Nov 25 03:36:45.690: INFO: cronjob-test-27822447-x4zbs started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container test ready: false, restart count 0
Nov 25 03:36:45.690: INFO: metadata-proxy-v0.1-pcvjx started at 2022-11-25 03:01:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:45.690: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:45.690: INFO: coredns-6d97d5ddb-ts7gx started at 2022-11-25 03:01:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container coredns ready: false, restart count 10
Nov 25 03:36:45.690: INFO: nfs-server started at 2022-11-25 03:09:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container nfs-server ready: false, restart count 8
Nov 25 03:36:45.690: INFO: agnhost-replica-74c659fd6f-kb9c5 started at 2022-11-25 03:25:52 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container replica ready: false, restart count 6
Nov 25 03:36:45.690: INFO: coredns-6d97d5ddb-r9926 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container coredns ready: false, restart count 10
Nov 25 03:36:45.690: INFO: ss2-1 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container webserver ready: false, restart count 6
Nov 25 03:36:45.690: INFO: recycler-for-nfs-h8mdt started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container pv-recycler ready: false, restart count 0
Nov 25 03:36:45.690: INFO: external-provisioner-2xvfz started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container nfs-provisioner ready: false, restart count 0
Nov 25 03:36:45.690: INFO: pvc-volume-tester-s584h started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:45.690: INFO: cronjob-test-27822446-bc7vn started at 2022-11-25 03:26:00 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container test ready: false, restart count 0
Nov 25 03:36:45.690: INFO: frontend-78cd9dfff5-47dtr started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container guestbook-frontend ready: false, restart count 5
Nov 25 03:36:45.690: INFO: pod-projected-configmaps-d14446ce-85cc-427e-8b6c-70f12a4cf35e started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container agnhost-container ready: false, restart count 0
Nov 25 03:36:45.690: INFO: volume-snapshot-controller-0 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container volume-snapshot-controller ready: true, restart count 11
Nov 25 03:36:45.690: INFO: konnectivity-agent-gd8tk started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container konnectivity-agent ready: false, restart count 7
Nov 25 03:36:45.690: INFO: csi-mockplugin-resizer-0 started at 2022-11-25 03:33:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container csi-resizer ready: true, restart count 1
Nov 25 03:36:45.690: INFO: netserver-2 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container webserver ready: false, restart count 4
Nov 25 03:36:45.690: INFO: external-provisioner-xxh24 started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container nfs-provisioner ready: true, restart count 4
Nov 25 03:36:45.690: INFO: startup-041d0114-e067-411d-af3e-f08e5df4b105 started at 2022-11-25 03:33:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container busybox ready: false, restart count 3
Nov 25 03:36:45.690: INFO: pod-size-memory-volume-cbc06427-7afe-4841-b4e8-9dae9feddc6a started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container busybox-main-container ready: false, restart count 0
Nov 25 03:36:45.690: INFO: external-provisioner-68w4b started at 2022-11-25 03:25:46 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container nfs-provisioner ready: false, restart count 5
Nov 25 03:36:45.690: INFO: kube-proxy-bootstrap-e2e-minion-group-q7zm started at 2022-11-25 03:01:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container kube-proxy ready: false, restart count 9
Nov 25 03:36:45.690: INFO: csi-mockplugin-0 started at 2022-11-25 03:11:56 +0000 UTC (0+4 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container busybox ready: false, restart count 7
Nov 25 03:36:45.690: INFO: 	Container csi-provisioner ready: true, restart count 8
Nov 25 03:36:45.690: INFO: 	Container driver-registrar ready: false, restart count 6
Nov 25 03:36:45.690: INFO: 	Container mock ready: false, restart count 6
Nov 25 03:36:45.690: INFO: service-headless-rxxnf started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container service-headless ready: true, restart count 2
Nov 25 03:36:45.690: INFO: pod-projected-secrets-fd485c27-166a-49eb-b3cc-a98f3b0470fa started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container projected-secret-volume-test ready: false, restart count 0
Nov 25 03:36:45.690: INFO: l7-default-backend-8549d69d99-k696w started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container default-http-backend ready: true, restart count 0
Nov 25 03:36:45.690: INFO: csi-mockplugin-0 started at 2022-11-25 03:33:33 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:36:45.690: INFO: 	Container driver-registrar ready: false, restart count 4
Nov 25 03:36:45.690: INFO: 	Container mock ready: false, restart count 4
Nov 25 03:36:45.690: INFO: external-provisioner-kgbhb started at 2022-11-25 03:33:35 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container nfs-provisioner ready: false, restart count 3
Nov 25 03:36:45.690: INFO: downwardapi-volume-927901cf-394f-4333-8c62-7d60bf4ee648 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:36:45.690: INFO: my-hostname-private-0c72cc77-0ea9-4ace-85e9-86c027c29230-6x65r started at 2022-11-25 03:25:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container my-hostname-private-0c72cc77-0ea9-4ace-85e9-86c027c29230 ready: true, restart count 6
Nov 25 03:36:45.690: INFO: kube-dns-autoscaler-5f6455f985-77mxq started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container autoscaler ready: false, restart count 9
Nov 25 03:36:45.690: INFO: pod-secrets-0a07040a-2718-4694-83b9-20611c00d694 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:45.690: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 25 03:36:45.690: INFO: 	Container dels-volume-test ready: false, restart count 0
Nov 25 03:36:45.690: INFO: 	Container upds-volume-test ready: false, restart count 0
Nov 25 03:36:46.046: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-q7zm
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceDefinition resources [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "custom-resource-definition-7673" for this suite. 11/25/22 03:36:46.047

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] updates the published spec when one version gets renamed [Conformance] 1m49s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourcePublishOpenAPI\s\[Privileged\:ClusterAdmin\]\supdates\sthe\spublished\sspec\swhen\sone\sversion\sgets\srenamed\s\[Conformance\]$'

test/e2e/apimachinery/crd_publish_openapi.go:411
k8s.io/kubernetes/test/e2e/apimachinery.glob..func6.8()
	test/e2e/apimachinery/crd_publish_openapi.go:411 +0x3bb
				[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:18:41.727
Nov 25 03:18:41.727: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename crd-publish-openapi 11/25/22 03:18:41.729
STEP: Waiting for a default service account to be provisioned in namespace 11/25/22 03:19:32.86
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/25/22 03:19:32.945
[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:31
[It] updates the published spec when one version gets renamed [Conformance]
  test/e2e/apimachinery/crd_publish_openapi.go:391
STEP: set up a multi version CRD 11/25/22 03:19:34.777
Nov 25 03:19:34.778: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: rename a version 11/25/22 03:19:46.01
Nov 25 03:20:20.135: FAIL: Timeout: request did not complete within requested timeout - context deadline exceeded

Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.glob..func6.8()
	test/e2e/apimachinery/crd_publish_openapi.go:411 +0x3bb
[AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:20:20.135: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:20:28.952
STEP: Collecting events from namespace "crd-publish-openapi-860". 11/25/22 03:20:28.952
STEP: Found 0 events. 11/25/22 03:20:28.993
Nov 25 03:20:29.035: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:20:29.035: INFO: 
Nov 25 03:20:29.079: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:20:29.121: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 14600 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:08 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:20:29.121: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:20:29.165: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:20:29.241: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container konnectivity-server-container ready: true, restart count 1
Nov 25 03:20:29.241: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container kube-apiserver ready: false, restart count 1
Nov 25 03:20:29.241: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container kube-controller-manager ready: false, restart count 6
Nov 25 03:20:29.241: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container kube-scheduler ready: false, restart count 5
Nov 25 03:20:29.241: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container kube-addon-manager ready: true, restart count 4
Nov 25 03:20:29.241: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container l7-lb-controller ready: false, restart count 6
Nov 25 03:20:29.241: INFO: metadata-proxy-v0.1-vkhjm started at 2022-11-25 03:01:26 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:20:29.241: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:20:29.241: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container etcd-container ready: true, restart count 0
Nov 25 03:20:29.241: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.241: INFO: 	Container etcd-container ready: true, restart count 4
Nov 25 03:20:29.484: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 25 03:20:29.484: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:20:29.535: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 14972 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2480":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:11:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:19:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:19:34 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-9455^bb2344cc-6c6f-11ed-b8ba-ae517b0b92db],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:20:29.535: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:20:29.583: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:20:29.765: INFO: pod-with-poststart-https-hook started at 2022-11-25 03:19:43 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container pod-with-poststart-https-hook ready: true, restart count 1
Nov 25 03:20:29.765: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:03:12 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container csi-attacher ready: false, restart count 6
Nov 25 03:20:29.765: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 25 03:20:29.765: INFO: 	Container csi-resizer ready: false, restart count 6
Nov 25 03:20:29.765: INFO: 	Container csi-snapshotter ready: false, restart count 6
Nov 25 03:20:29.765: INFO: 	Container hostpath ready: false, restart count 6
Nov 25 03:20:29.765: INFO: 	Container liveness-probe ready: false, restart count 6
Nov 25 03:20:29.765: INFO: 	Container node-driver-registrar ready: false, restart count 6
Nov 25 03:20:29.765: INFO: hostpath-injector started at 2022-11-25 03:19:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container hostpath-injector ready: false, restart count 0
Nov 25 03:20:29.765: INFO: kube-proxy-bootstrap-e2e-minion-group-2842 started at 2022-11-25 03:01:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container kube-proxy ready: false, restart count 6
Nov 25 03:20:29.765: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:09:21 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container csi-attacher ready: true, restart count 4
Nov 25 03:20:29.765: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 25 03:20:29.765: INFO: 	Container csi-resizer ready: true, restart count 4
Nov 25 03:20:29.765: INFO: 	Container csi-snapshotter ready: true, restart count 4
Nov 25 03:20:29.765: INFO: 	Container hostpath ready: true, restart count 4
Nov 25 03:20:29.765: INFO: 	Container liveness-probe ready: true, restart count 4
Nov 25 03:20:29.765: INFO: 	Container node-driver-registrar ready: true, restart count 4
Nov 25 03:20:29.765: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:16:00 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container csi-attacher ready: false, restart count 5
Nov 25 03:20:29.765: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 25 03:20:29.765: INFO: 	Container csi-resizer ready: false, restart count 5
Nov 25 03:20:29.765: INFO: 	Container csi-snapshotter ready: false, restart count 5
Nov 25 03:20:29.765: INFO: 	Container hostpath ready: false, restart count 5
Nov 25 03:20:29.765: INFO: 	Container liveness-probe ready: false, restart count 5
Nov 25 03:20:29.765: INFO: 	Container node-driver-registrar ready: false, restart count 5
Nov 25 03:20:29.765: INFO: sample-webhook-deployment-865554f4d9-s2mvw started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container sample-webhook ready: true, restart count 3
Nov 25 03:20:29.765: INFO: pod-server-1 started at 2022-11-25 03:15:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:20:29.765: INFO: ss2-0 started at 2022-11-25 03:15:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container webserver ready: false, restart count 2
Nov 25 03:20:29.765: INFO: konnectivity-agent-xr8pf started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container konnectivity-agent ready: true, restart count 7
Nov 25 03:20:29.765: INFO: netserver-0 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container webserver ready: false, restart count 4
Nov 25 03:20:29.765: INFO: pod-handle-http-request started at 2022-11-25 03:19:40 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container container-handle-http-request ready: true, restart count 0
Nov 25 03:20:29.765: INFO: 	Container container-handle-https-request ready: true, restart count 0
Nov 25 03:20:29.765: INFO: metadata-proxy-v0.1-4nskz started at 2022-11-25 03:01:32 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:20:29.765: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:20:29.765: INFO: metrics-server-v0.5.2-867b8754b9-mkqv8 started at 2022-11-25 03:01:55 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container metrics-server ready: false, restart count 6
Nov 25 03:20:29.765: INFO: 	Container metrics-server-nanny ready: false, restart count 7
Nov 25 03:20:29.765: INFO: hostexec-bootstrap-e2e-minion-group-2842-khxh6 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:20:29.765: INFO: test-deployment-tw5kw-54bc444df-5lzn2 started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:29.765: INFO: 	Container httpd ready: true, restart count 3
Nov 25 03:20:30.118: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-2842
Nov 25 03:20:30.118: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:20:30.161: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 14898 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1494":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-1554":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:19:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {node-problem-detector Update v1 2022-11-25 03:19:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:19:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:20:30.162: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:20:30.205: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:20:30.272: INFO: kube-proxy-bootstrap-e2e-minion-group-4j20 started at 2022-11-25 03:01:30 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container kube-proxy ready: false, restart count 7
Nov 25 03:20:30.272: INFO: dns-test-fa248686-6c83-425f-834a-b5c24a5cd609 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container jessie-querier ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container querier ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container webserver ready: true, restart count 3
Nov 25 03:20:30.272: INFO: execpodndnps started at 2022-11-25 03:15:48 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:20:30.272: INFO: agnhost started at 2022-11-25 03:03:43 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container agnhost ready: false, restart count 0
Nov 25 03:20:30.272: INFO: netserver-1 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container webserver ready: true, restart count 4
Nov 25 03:20:30.272: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:09:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 25 03:20:30.272: INFO: host-test-container-pod started at 2022-11-25 03:12:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container agnhost-container ready: false, restart count 2
Nov 25 03:20:30.272: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:13:23 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:20:30.272: INFO: pvc-volume-tester-5snsv started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:20:30.272: INFO: ss2-2 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container webserver ready: true, restart count 1
Nov 25 03:20:30.272: INFO: pod2 started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 25 03:20:30.272: INFO: csi-mockplugin-0 started at 2022-11-25 03:09:45 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 25 03:20:30.272: INFO: 	Container driver-registrar ready: true, restart count 4
Nov 25 03:20:30.272: INFO: 	Container mock ready: true, restart count 4
Nov 25 03:20:30.272: INFO: konnectivity-agent-wbnhz started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container konnectivity-agent ready: false, restart count 7
Nov 25 03:20:30.272: INFO: metadata-proxy-v0.1-s4hwh started at 2022-11-25 03:01:31 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:20:30.272: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:20:30.272: INFO: test-container-pod started at 2022-11-25 03:12:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container webserver ready: true, restart count 4
Nov 25 03:20:30.272: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:15:52 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container csi-provisioner ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container csi-resizer ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container csi-snapshotter ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container hostpath ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container liveness-probe ready: true, restart count 3
Nov 25 03:20:30.272: INFO: 	Container node-driver-registrar ready: true, restart count 3
Nov 25 03:20:30.272: INFO: pod-subpath-test-dynamicpv-xn2k started at 2022-11-25 03:15:57 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container test-container-subpath-dynamicpv-xn2k ready: false, restart count 0
Nov 25 03:20:30.272: INFO: 	Container test-container-volume-dynamicpv-xn2k ready: false, restart count 0
Nov 25 03:20:30.272: INFO: csi-mockplugin-0 started at 2022-11-25 03:13:23 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:20:30.272: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:20:30.272: INFO: 	Container driver-registrar ready: true, restart count 1
Nov 25 03:20:30.272: INFO: 	Container mock ready: true, restart count 1
Nov 25 03:20:30.628: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-4j20
Nov 25 03:20:30.628: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:20:30.670: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 14849 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-11-25 03:19:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:20:30.670: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:20:30.717: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:20:30.796: INFO: recycler-for-nfs-h8mdt started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container pv-recycler ready: false, restart count 0
Nov 25 03:20:30.796: INFO: nfs-server started at 2022-11-25 03:19:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container nfs-server ready: true, restart count 0
Nov 25 03:20:30.796: INFO: volume-snapshot-controller-0 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container volume-snapshot-controller ready: false, restart count 6
Nov 25 03:20:30.796: INFO: konnectivity-agent-gd8tk started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container konnectivity-agent ready: false, restart count 5
Nov 25 03:20:30.796: INFO: pvc-volume-tester-s584h started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:20:30.796: INFO: hostexec-bootstrap-e2e-minion-group-q7zm-zmx74 started at 2022-11-25 03:19:38 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container agnhost-container ready: true, restart count 0
Nov 25 03:20:30.796: INFO: nfs-server started at 2022-11-25 03:19:40 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container nfs-server ready: true, restart count 2
Nov 25 03:20:30.796: INFO: nfs-server started at 2022-11-25 03:19:34 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container nfs-server ready: true, restart count 0
Nov 25 03:20:30.796: INFO: kube-proxy-bootstrap-e2e-minion-group-q7zm started at 2022-11-25 03:01:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container kube-proxy ready: false, restart count 7
Nov 25 03:20:30.796: INFO: csi-mockplugin-0 started at 2022-11-25 03:11:56 +0000 UTC (0+4 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container busybox ready: false, restart count 4
Nov 25 03:20:30.796: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 25 03:20:30.796: INFO: 	Container driver-registrar ready: false, restart count 3
Nov 25 03:20:30.796: INFO: 	Container mock ready: false, restart count 3
Nov 25 03:20:30.796: INFO: pod-projected-secrets-fd485c27-166a-49eb-b3cc-a98f3b0470fa started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container projected-secret-volume-test ready: false, restart count 0
Nov 25 03:20:30.796: INFO: l7-default-backend-8549d69d99-k696w started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container default-http-backend ready: true, restart count 0
Nov 25 03:20:30.796: INFO: execpodlc59j started at 2022-11-25 03:13:12 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container agnhost-container ready: true, restart count 5
Nov 25 03:20:30.796: INFO: netserver-2 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:20:30.796: INFO: downwardapi-volume-927901cf-394f-4333-8c62-7d60bf4ee648 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:20:30.796: INFO: pod-init-4290d50d-1359-4af5-864f-541bf267bec2 started at 2022-11-25 03:19:38 +0000 UTC (2+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Init container init1 ready: true, restart count 0
Nov 25 03:20:30.796: INFO: 	Init container init2 ready: false, restart count 0
Nov 25 03:20:30.796: INFO: 	Container run1 ready: false, restart count 0
Nov 25 03:20:30.796: INFO: external-provisioner-9zppw started at 2022-11-25 03:19:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container nfs-provisioner ready: true, restart count 2
Nov 25 03:20:30.796: INFO: kube-dns-autoscaler-5f6455f985-77mxq started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container autoscaler ready: false, restart count 7
Nov 25 03:20:30.796: INFO: pod-secrets-0a07040a-2718-4694-83b9-20611c00d694 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 25 03:20:30.796: INFO: 	Container dels-volume-test ready: false, restart count 0
Nov 25 03:20:30.796: INFO: 	Container upds-volume-test ready: false, restart count 0
Nov 25 03:20:30.796: INFO: pod-handle-http-request started at 2022-11-25 03:15:49 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container container-handle-http-request ready: true, restart count 1
Nov 25 03:20:30.796: INFO: 	Container container-handle-https-request ready: true, restart count 1
Nov 25 03:20:30.796: INFO: test-runtimeclass-runtimeclass-6746-unconfigured-handler-7qdnz started at 2022-11-25 03:19:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container test ready: false, restart count 0
Nov 25 03:20:30.796: INFO: pod1 started at 2022-11-25 03:13:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:20:30.796: INFO: metadata-proxy-v0.1-pcvjx started at 2022-11-25 03:01:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:20:30.796: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:20:30.796: INFO: coredns-6d97d5ddb-ts7gx started at 2022-11-25 03:01:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container coredns ready: false, restart count 7
Nov 25 03:20:30.796: INFO: pod-configmaps-d4ab1bf2-e5b0-4eb4-96aa-d9c8f5ebcd8f started at 2022-11-25 03:19:39 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container agnhost-container ready: true, restart count 0
Nov 25 03:20:30.796: INFO: coredns-6d97d5ddb-r9926 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container coredns ready: false, restart count 7
Nov 25 03:20:30.796: INFO: ss2-1 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container webserver ready: true, restart count 0
Nov 25 03:20:30.796: INFO: nfs-server started at 2022-11-25 03:09:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:20:30.796: INFO: 	Container nfs-server ready: false, restart count 6
Nov 25 03:20:31.075: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-q7zm
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "crd-publish-openapi-860" for this suite. 11/25/22 03:20:31.075

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD without validation schema [Conformance] 30s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourcePublishOpenAPI\s\[Privileged\:ClusterAdmin\]\sworks\sfor\sCRD\swithout\svalidation\sschema\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0002f6960)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:33:45.914
Nov 25 03:33:45.914: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename crd-publish-openapi 11/25/22 03:33:45.917
Nov 25 03:33:45.957: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:47.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:49.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:51.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:53.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:55.996: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:57.996: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:59.996: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:01.996: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:03.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:05.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:07.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:09.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:11.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:13.996: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:15.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:16.036: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:16.036: INFO: Unexpected error: 
    <*errors.errorString | 0xc000115d80>: {
        s: "timed out waiting for the condition",
    }
Nov 25 03:34:16.036: FAIL: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0002f6960)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:34:16.037: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:34:16.076
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  tear down framework | framework.go:193

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for multiple CRDs of same group and version but different kinds [Conformance] 2m0s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourcePublishOpenAPI\s\[Privileged\:ClusterAdmin\]\sworks\sfor\smultiple\sCRDs\sof\ssame\sgroup\sand\sversion\sbut\sdifferent\skinds\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0008e8960)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:20:31.632
Nov 25 03:20:31.632: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename crd-publish-openapi 11/25/22 03:20:31.634
Nov 25 03:22:31.681: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc0002fc060>: {
        msg: "wait for service account \"default\" in namespace \"crd-publish-openapi-8138\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc0001ffa10>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:22:31.681: FAIL: wait for service account "default" in namespace "crd-publish-openapi-8138": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0008e8960)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:22:31.682: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:22:31.725
STEP: Collecting events from namespace "crd-publish-openapi-8138". 11/25/22 03:22:31.725
STEP: Found 0 events. 11/25/22 03:22:31.767
Nov 25 03:22:31.807: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:22:31.807: INFO: 
Nov 25 03:22:31.853: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:22:31.900: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 15645 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:22:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:22:31.901: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:22:31.946: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:22:31.988: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:22:31.988: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:22:32.030: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 15619 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2480":"bootstrap-e2e-minion-group-2842","csi-hostpath-volume-9455":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:11:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:19:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:22:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-9455^bb2344cc-6c6f-11ed-b8ba-ae517b0b92db],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:22:32.030: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:22:32.073: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:22:32.116: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:22:32.116: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:22:32.159: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 15660 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1494":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:19:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:19:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:22:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:22:32.160: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:22:32.203: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:22:32.245: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:22:32.245: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:22:32.287: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 14849 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-11-25 03:19:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:22:32.287: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:22:32.333: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:22:32.382: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "crd-publish-openapi-8138" for this suite. 11/25/22 03:22:32.382

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST NOT fail validation for create of a custom resource that satisfies the x-kubernetes-validations rules 2m0s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceValidationRules\s\[Privileged\:ClusterAdmin\]\sMUST\sNOT\sfail\svalidation\sfor\screate\sof\sa\scustom\sresource\sthat\ssatisfies\sthe\sx\-kubernetes\-validations\srules$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000378a50)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:20:59.999
Nov 25 03:20:59.999: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename crd-validation-expressions 11/25/22 03:21:00.001
Nov 25 03:23:00.048: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc000cda000>: {
        msg: "wait for service account \"default\" in namespace \"crd-validation-expressions-8092\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc000115d80>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:23:00.048: FAIL: wait for service account "default" in namespace "crd-validation-expressions-8092": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000378a50)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:23:00.048: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:23:00.091
STEP: Collecting events from namespace "crd-validation-expressions-8092". 11/25/22 03:23:00.091
STEP: Found 0 events. 11/25/22 03:23:00.131
Nov 25 03:23:00.172: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:23:00.172: INFO: 
Nov 25 03:23:00.215: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:23:00.257: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 15645 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:22:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:22:14 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:23:00.257: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:23:00.301: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:23:00.344: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-master: error trying to reach service: No agent available
Nov 25 03:23:00.344: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:23:00.385: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 15619 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2480":"bootstrap-e2e-minion-group-2842","csi-hostpath-volume-9455":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:11:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:19:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:22:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:19:34 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-volume-9455^bb2344cc-6c6f-11ed-b8ba-ae517b0b92db],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:23:00.386: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:23:00.432: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:23:00.474: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-2842: error trying to reach service: No agent available
Nov 25 03:23:00.474: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:23:00.517: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 15660 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1494":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:19:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:19:32 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:22:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:21:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:23:00.517: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:23:00.561: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:23:00.603: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-4j20: error trying to reach service: No agent available
Nov 25 03:23:00.603: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:23:00.644: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 15706 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:19:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:22:38 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:22:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:22:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:23:00.645: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:23:00.688: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:23:00.731: INFO: Unable to retrieve kubelet pods for node bootstrap-e2e-minion-group-q7zm: error trying to reach service: No agent available
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  tear down framework | framework.go:193
STEP: Destroying namespace "crd-validation-expressions-8092" for this suite. 11/25/22 03:23:00.732

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin] MUST fail create of a custom resource definition that contains an x-kubernetes-validations rule that exceeds the estimated cost limit 30s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sCustomResourceValidationRules\s\[Privileged\:ClusterAdmin\]\sMUST\sfail\screate\sof\sa\scustom\sresource\sdefinition\sthat\scontains\san\sx\-kubernetes\-validations\srule\sthat\sexceeds\sthe\sestimated\scost\slimit$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00109ea50)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:33:45.913
Nov 25 03:33:45.913: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename crd-validation-expressions 11/25/22 03:33:45.915
Nov 25 03:33:45.954: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:47.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:49.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:51.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:53.993: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:55.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:57.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:33:59.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:01.995: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:03.995: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:05.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:07.997: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:09.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:11.995: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:13.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:15.994: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:16.034: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:16.034: INFO: Unexpected error: 
    <*errors.errorString | 0xc000295c50>: {
        s: "timed out waiting for the condition",
    }
Nov 25 03:34:16.034: FAIL: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc00109ea50)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  test/e2e/framework/node/init/init.go:32
Nov 25 03:34:16.034: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:34:16.073
[DeferCleanup (Each)] [sig-api-machinery] CustomResourceValidationRules [Privileged:ClusterAdmin]
  tear down framework | framework.go:193

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should delete jobs and pods created by cronjob 2m30s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sGarbage\scollector\sshould\sdelete\sjobs\sand\spods\screated\sby\scronjob$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0007f8ff0)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] Garbage collector
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:34:15.475
Nov 25 03:34:15.475: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename gc 11/25/22 03:34:15.478
Nov 25 03:34:15.517: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:17.557: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:19.558: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:21.557: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:23.557: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:25.557: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:27.557: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:29.557: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:34:31.558: INFO: Unexpected error while creating namespace: Post "https://35.247.30.25/api/v1/namespaces": dial tcp 35.247.30.25:443: connect: connection refused
Nov 25 03:36:36.722: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc000ca6120>: {
        msg: "wait for service account \"default\" in namespace \"gc-3772\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc00017fa10>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:36:36.723: FAIL: wait for service account "default" in namespace "gc-3772": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0007f8ff0)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] Garbage collector
  test/e2e/framework/node/init/init.go:32
Nov 25 03:36:36.723: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:36:36.816
STEP: Collecting events from namespace "gc-3772". 11/25/22 03:36:36.816
STEP: Found 0 events. 11/25/22 03:36:36.858
Nov 25 03:36:36.904: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:36:36.904: INFO: 
Nov 25 03:36:36.954: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:36:37.014: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 19366 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:32:28 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:32:28 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:36:37.015: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:36:37.092: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:36:37.356: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container konnectivity-server-container ready: true, restart count 1
Nov 25 03:36:37.356: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container kube-controller-manager ready: false, restart count 8
Nov 25 03:36:37.356: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container kube-scheduler ready: false, restart count 7
Nov 25 03:36:37.356: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container kube-addon-manager ready: true, restart count 5
Nov 25 03:36:37.356: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container l7-lb-controller ready: true, restart count 10
Nov 25 03:36:37.356: INFO: metadata-proxy-v0.1-vkhjm started at 2022-11-25 03:01:26 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:37.356: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:37.356: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container etcd-container ready: true, restart count 0
Nov 25 03:36:37.356: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container etcd-container ready: true, restart count 4
Nov 25 03:36:37.356: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:37.356: INFO: 	Container kube-apiserver ready: true, restart count 2
Nov 25 03:36:38.538: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 25 03:36:38.538: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:36:38.669: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 20744 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-3961":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:33:19 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:34:38 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:11 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:38 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:34:43 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-3961^e6de85f2-6c71-11ed-9fff-d6801b68d09f,DevicePath:,},},Config:nil,},}
Nov 25 03:36:38.669: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:36:38.812: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:36:39.528: INFO: konnectivity-agent-xr8pf started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container konnectivity-agent ready: true, restart count 10
Nov 25 03:36:39.528: INFO: ss2-0 started at 2022-11-25 03:15:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container webserver ready: false, restart count 7
Nov 25 03:36:39.528: INFO: agnhost-primary-75659d4b45-ppxww started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container primary ready: false, restart count 5
Nov 25 03:36:39.528: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 25 03:36:39.528: INFO: hostexec-bootstrap-e2e-minion-group-2842-8q8zg started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container agnhost-container ready: true, restart count 1
Nov 25 03:36:39.528: INFO: metadata-proxy-v0.1-4nskz started at 2022-11-25 03:01:32 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:39.528: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:39.528: INFO: simpletest.rc-hp82q started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container nginx ready: true, restart count 0
Nov 25 03:36:39.528: INFO: pod-ready started at 2022-11-25 03:25:48 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container pod-readiness-gate ready: true, restart count 4
Nov 25 03:36:39.528: INFO: oidc-discovery-validator started at 2022-11-25 03:25:48 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container oidc-discovery-validator ready: false, restart count 0
Nov 25 03:36:39.528: INFO: metrics-server-v0.5.2-867b8754b9-mkqv8 started at 2022-11-25 03:01:55 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container metrics-server ready: false, restart count 9
Nov 25 03:36:39.528: INFO: 	Container metrics-server-nanny ready: false, restart count 10
Nov 25 03:36:39.528: INFO: agnhost-replica-74c659fd6f-hkm8d started at 2022-11-25 03:25:52 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container replica ready: false, restart count 6
Nov 25 03:36:39.528: INFO: service-headless-wfqcn started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container service-headless ready: true, restart count 3
Nov 25 03:36:39.528: INFO: hostexec-bootstrap-e2e-minion-group-2842-6lzrc started at 2022-11-25 03:33:35 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container agnhost-container ready: true, restart count 5
Nov 25 03:36:39.528: INFO: hostexec-bootstrap-e2e-minion-group-2842-khxh6 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container agnhost-container ready: false, restart count 6
Nov 25 03:36:39.528: INFO: netserver-0 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:36:39.528: INFO: boom-server started at 2022-11-25 03:33:27 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container boom-server ready: true, restart count 1
Nov 25 03:36:39.528: INFO: csi-mockplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:36:39.528: INFO: 	Container driver-registrar ready: true, restart count 1
Nov 25 03:36:39.528: INFO: 	Container mock ready: true, restart count 1
Nov 25 03:36:39.528: INFO: frontend-78cd9dfff5-nj8mr started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container guestbook-frontend ready: false, restart count 6
Nov 25 03:36:39.528: INFO: pvc-volume-tester-w5szh started at 2022-11-25 03:33:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:39.528: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:03:12 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container csi-attacher ready: false, restart count 10
Nov 25 03:36:39.528: INFO: 	Container csi-provisioner ready: false, restart count 10
Nov 25 03:36:39.528: INFO: 	Container csi-resizer ready: false, restart count 10
Nov 25 03:36:39.528: INFO: 	Container csi-snapshotter ready: false, restart count 10
Nov 25 03:36:39.528: INFO: 	Container hostpath ready: false, restart count 10
Nov 25 03:36:39.528: INFO: 	Container liveness-probe ready: false, restart count 10
Nov 25 03:36:39.528: INFO: 	Container node-driver-registrar ready: false, restart count 10
Nov 25 03:36:39.528: INFO: kube-proxy-bootstrap-e2e-minion-group-2842 started at 2022-11-25 03:01:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container kube-proxy ready: false, restart count 9
Nov 25 03:36:39.528: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:09:21 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container csi-attacher ready: false, restart count 6
Nov 25 03:36:39.528: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 25 03:36:39.528: INFO: 	Container csi-resizer ready: false, restart count 6
Nov 25 03:36:39.528: INFO: 	Container csi-snapshotter ready: false, restart count 6
Nov 25 03:36:39.528: INFO: 	Container hostpath ready: false, restart count 6
Nov 25 03:36:39.528: INFO: 	Container liveness-probe ready: false, restart count 6
Nov 25 03:36:39.528: INFO: 	Container node-driver-registrar ready: false, restart count 6
Nov 25 03:36:39.528: INFO: sample-webhook-deployment-865554f4d9-s2mvw started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container sample-webhook ready: false, restart count 6
Nov 25 03:36:39.528: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:16:00 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container csi-attacher ready: false, restart count 8
Nov 25 03:36:39.528: INFO: 	Container csi-provisioner ready: false, restart count 8
Nov 25 03:36:39.528: INFO: 	Container csi-resizer ready: false, restart count 8
Nov 25 03:36:39.528: INFO: 	Container csi-snapshotter ready: false, restart count 8
Nov 25 03:36:39.528: INFO: 	Container hostpath ready: false, restart count 8
Nov 25 03:36:39.528: INFO: 	Container liveness-probe ready: false, restart count 8
Nov 25 03:36:39.528: INFO: 	Container node-driver-registrar ready: false, restart count 8
Nov 25 03:36:39.528: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:33:38 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:39.528: INFO: 	Container csi-attacher ready: false, restart count 4
Nov 25 03:36:39.528: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:36:39.528: INFO: 	Container csi-resizer ready: false, restart count 4
Nov 25 03:36:39.528: INFO: 	Container csi-snapshotter ready: false, restart count 4
Nov 25 03:36:39.528: INFO: 	Container hostpath ready: false, restart count 4
Nov 25 03:36:39.528: INFO: 	Container liveness-probe ready: false, restart count 4
Nov 25 03:36:39.528: INFO: 	Container node-driver-registrar ready: false, restart count 4
Nov 25 03:36:44.775: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-2842
Nov 25 03:36:44.775: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:44.817: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 20813 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-mock-csi-mock-volumes-2265":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:26:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:34:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:31:53 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:36:44.817: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:44.873: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:44.952: INFO: konnectivity-agent-wbnhz started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.952: INFO: 	Container konnectivity-agent ready: false, restart count 9
Nov 25 03:36:44.952: INFO: metadata-proxy-v0.1-s4hwh started at 2022-11-25 03:01:31 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:44.952: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:44.952: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:44.952: INFO: pod-e66b7f89-d8d9-46f2-90a6-f9d1fc4a6523 started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.952: INFO: 	Container test-container ready: false, restart count 0
Nov 25 03:36:44.952: INFO: pvc-volume-tester-hk7qj started at 2022-11-25 03:33:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.952: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:44.952: INFO: csi-mockplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:44.952: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:36:44.952: INFO: 	Container driver-registrar ready: true, restart count 1
Nov 25 03:36:44.952: INFO: 	Container mock ready: true, restart count 1
Nov 25 03:36:44.952: INFO: csi-mockplugin-0 started at 2022-11-25 03:13:23 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 25 03:36:44.953: INFO: 	Container driver-registrar ready: false, restart count 5
Nov 25 03:36:44.953: INFO: 	Container mock ready: false, restart count 5
Nov 25 03:36:44.953: INFO: kube-proxy-bootstrap-e2e-minion-group-4j20 started at 2022-11-25 03:01:30 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container kube-proxy ready: true, restart count 10
Nov 25 03:36:44.953: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container csi-attacher ready: false, restart count 1
Nov 25 03:36:44.953: INFO: 	Container csi-provisioner ready: false, restart count 1
Nov 25 03:36:44.953: INFO: 	Container csi-resizer ready: false, restart count 1
Nov 25 03:36:44.953: INFO: 	Container csi-snapshotter ready: false, restart count 1
Nov 25 03:36:44.953: INFO: 	Container hostpath ready: false, restart count 1
Nov 25 03:36:44.953: INFO: 	Container liveness-probe ready: false, restart count 1
Nov 25 03:36:44.953: INFO: 	Container node-driver-registrar ready: false, restart count 1
Nov 25 03:36:44.953: INFO: hostexec-bootstrap-e2e-minion-group-4j20-ffbwm started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:36:44.953: INFO: annotationupdateb256174a-aa90-4993-b6eb-102898f10910 started at 2022-11-25 03:33:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:36:44.953: INFO: simpletest.rc-89gpg started at 2022-11-25 03:33:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container nginx ready: true, restart count 1
Nov 25 03:36:44.953: INFO: netserver-1 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container webserver ready: false, restart count 4
Nov 25 03:36:44.953: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:09:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container csi-attacher ready: false, restart count 5
Nov 25 03:36:44.953: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:13:23 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container csi-attacher ready: true, restart count 5
Nov 25 03:36:44.953: INFO: pvc-volume-tester-5snsv started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:44.953: INFO: ss2-2 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container webserver ready: false, restart count 6
Nov 25 03:36:44.953: INFO: pfpod started at 2022-11-25 03:33:28 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container portforwardtester ready: false, restart count 0
Nov 25 03:36:44.953: INFO: 	Container readiness ready: false, restart count 0
Nov 25 03:36:44.953: INFO: service-headless-6w72j started at 2022-11-25 03:33:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container service-headless ready: true, restart count 3
Nov 25 03:36:44.953: INFO: frontend-78cd9dfff5-8ttbp started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container guestbook-frontend ready: false, restart count 5
Nov 25 03:36:44.953: INFO: pod-handle-http-request started at 2022-11-25 03:33:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container container-handle-http-request ready: true, restart count 2
Nov 25 03:36:44.953: INFO: 	Container container-handle-https-request ready: true, restart count 2
Nov 25 03:36:44.953: INFO: csi-mockplugin-0 started at 2022-11-25 03:09:45 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:44.953: INFO: 	Container csi-provisioner ready: false, restart count 7
Nov 25 03:36:44.953: INFO: 	Container driver-registrar ready: false, restart count 7
Nov 25 03:36:44.953: INFO: 	Container mock ready: false, restart count 7
Nov 25 03:36:45.499: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-4j20
Nov 25 03:36:45.499: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:36:45.541: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 20729 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:34:44 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:36:05 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:34:44 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:36:05 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[gcr.io/k8s-authenticated-test/agnhost@sha256:4057a5580c7b59c4fe10d8ab2732c9dec35eea80fd41f7bafc7bd5acc7edf727 gcr.io/k8s-authenticated-test/agnhost:2.6],SizeBytes:18352698,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:36:45.541: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:36:45.585: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:36:45.662: INFO: csi-mockplugin-0 started at 2022-11-25 03:11:56 +0000 UTC (0+4 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container busybox ready: false, restart count 7
Nov 25 03:36:45.662: INFO: 	Container csi-provisioner ready: true, restart count 8
Nov 25 03:36:45.662: INFO: 	Container driver-registrar ready: false, restart count 6
Nov 25 03:36:45.662: INFO: 	Container mock ready: false, restart count 6
Nov 25 03:36:45.662: INFO: pod-size-memory-volume-cbc06427-7afe-4841-b4e8-9dae9feddc6a started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container busybox-main-container ready: false, restart count 0
Nov 25 03:36:45.662: INFO: external-provisioner-68w4b started at 2022-11-25 03:25:46 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container nfs-provisioner ready: false, restart count 5
Nov 25 03:36:45.662: INFO: kube-proxy-bootstrap-e2e-minion-group-q7zm started at 2022-11-25 03:01:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container kube-proxy ready: false, restart count 9
Nov 25 03:36:45.662: INFO: service-headless-rxxnf started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container service-headless ready: true, restart count 2
Nov 25 03:36:45.662: INFO: pod-projected-secrets-fd485c27-166a-49eb-b3cc-a98f3b0470fa started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container projected-secret-volume-test ready: false, restart count 0
Nov 25 03:36:45.662: INFO: l7-default-backend-8549d69d99-k696w started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container default-http-backend ready: true, restart count 0
Nov 25 03:36:45.662: INFO: external-provisioner-kgbhb started at 2022-11-25 03:33:35 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container nfs-provisioner ready: false, restart count 3
Nov 25 03:36:45.662: INFO: csi-mockplugin-0 started at 2022-11-25 03:33:33 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:36:45.662: INFO: 	Container driver-registrar ready: false, restart count 4
Nov 25 03:36:45.662: INFO: 	Container mock ready: false, restart count 4
Nov 25 03:36:45.662: INFO: downwardapi-volume-927901cf-394f-4333-8c62-7d60bf4ee648 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:36:45.662: INFO: pod-secrets-0a07040a-2718-4694-83b9-20611c00d694 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 25 03:36:45.662: INFO: 	Container dels-volume-test ready: false, restart count 0
Nov 25 03:36:45.662: INFO: 	Container upds-volume-test ready: false, restart count 0
Nov 25 03:36:45.662: INFO: my-hostname-private-0c72cc77-0ea9-4ace-85e9-86c027c29230-6x65r started at 2022-11-25 03:25:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container my-hostname-private-0c72cc77-0ea9-4ace-85e9-86c027c29230 ready: true, restart count 6
Nov 25 03:36:45.662: INFO: kube-dns-autoscaler-5f6455f985-77mxq started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container autoscaler ready: false, restart count 9
Nov 25 03:36:45.662: INFO: cronjob-test-27822453-w744f started at 2022-11-25 03:33:17 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container test ready: false, restart count 0
Nov 25 03:36:45.662: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:33:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container csi-attacher ready: false, restart count 2
Nov 25 03:36:45.662: INFO: coredns-6d97d5ddb-ts7gx started at 2022-11-25 03:01:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container coredns ready: false, restart count 10
Nov 25 03:36:45.662: INFO: cronjob-test-27822447-x4zbs started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container test ready: false, restart count 0
Nov 25 03:36:45.662: INFO: metadata-proxy-v0.1-pcvjx started at 2022-11-25 03:01:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:36:45.662: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:36:45.662: INFO: ss2-1 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container webserver ready: false, restart count 6
Nov 25 03:36:45.662: INFO: nfs-server started at 2022-11-25 03:09:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container nfs-server ready: false, restart count 8
Nov 25 03:36:45.662: INFO: agnhost-replica-74c659fd6f-kb9c5 started at 2022-11-25 03:25:52 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container replica ready: false, restart count 6
Nov 25 03:36:45.662: INFO: coredns-6d97d5ddb-r9926 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container coredns ready: false, restart count 10
Nov 25 03:36:45.662: INFO: external-provisioner-2xvfz started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container nfs-provisioner ready: false, restart count 0
Nov 25 03:36:45.662: INFO: recycler-for-nfs-h8mdt started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container pv-recycler ready: false, restart count 0
Nov 25 03:36:45.662: INFO: konnectivity-agent-gd8tk started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container konnectivity-agent ready: false, restart count 7
Nov 25 03:36:45.662: INFO: pvc-volume-tester-s584h started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:36:45.662: INFO: cronjob-test-27822446-bc7vn started at 2022-11-25 03:26:00 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container test ready: false, restart count 0
Nov 25 03:36:45.662: INFO: frontend-78cd9dfff5-47dtr started at 2022-11-25 03:25:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container guestbook-frontend ready: false, restart count 5
Nov 25 03:36:45.662: INFO: pod-projected-configmaps-d14446ce-85cc-427e-8b6c-70f12a4cf35e started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container agnhost-container ready: false, restart count 0
Nov 25 03:36:45.662: INFO: volume-snapshot-controller-0 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container volume-snapshot-controller ready: true, restart count 11
Nov 25 03:36:45.662: INFO: netserver-2 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container webserver ready: false, restart count 4
Nov 25 03:36:45.662: INFO: csi-mockplugin-resizer-0 started at 2022-11-25 03:33:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container csi-resizer ready: true, restart count 1
Nov 25 03:36:45.662: INFO: startup-041d0114-e067-411d-af3e-f08e5df4b105 started at 2022-11-25 03:33:33 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container busybox ready: false, restart count 3
Nov 25 03:36:45.662: INFO: external-provisioner-xxh24 started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:36:45.662: INFO: 	Container nfs-provisioner ready: true, restart count 4
Nov 25 03:36:45.914: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-q7zm
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  tear down framework | framework.go:193
STEP: Destroying namespace "gc-3772" for this suite. 11/25/22 03:36:45.914

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should orphan pods created by rc if deleteOptions.OrphanDependents is nil 2m30s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sGarbage\scollector\sshould\sorphan\spods\screated\sby\src\sif\sdeleteOptions\.OrphanDependents\sis\snil$'

test/e2e/apimachinery/garbage_collector.go:475
k8s.io/kubernetes/test/e2e/apimachinery.glob..func13.3()
	test/e2e/apimachinery/garbage_collector.go:475 +0x4d1
There were additional failures detected after the initial failure:
[FAILED]
Nov 25 03:33:57.077: failed to list events in namespace "gc-1123": Get "https://35.247.30.25/api/v1/namespaces/gc-1123/events": dial tcp 35.247.30.25:443: connect: connection refused
In [DeferCleanup (Each)] at: test/e2e/framework/debug/dump.go:44
----------
[FAILED]
Nov 25 03:33:57.117: Couldn't delete ns: "gc-1123": Delete "https://35.247.30.25/api/v1/namespaces/gc-1123": dial tcp 35.247.30.25:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://35.247.30.25/api/v1/namespaces/gc-1123", Err:(*net.OpError)(0xc0050d4b40)})
In [DeferCleanup (Each)] at: test/e2e/framework/framework.go:370

				[BeforeEach] [sig-api-machinery] Garbage collector
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:31:26.322
Nov 25 03:31:26.323: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename gc 11/25/22 03:31:26.326
STEP: Waiting for a default service account to be provisioned in namespace 11/25/22 03:33:18.076
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/25/22 03:33:18.189
[BeforeEach] [sig-api-machinery] Garbage collector
  test/e2e/framework/metrics/init/init.go:31
[It] should orphan pods created by rc if deleteOptions.OrphanDependents is nil
  test/e2e/apimachinery/garbage_collector.go:439
STEP: create the rc 11/25/22 03:33:21.714
STEP: delete the rc 11/25/22 03:33:26.843
STEP: wait for 30 seconds to see if the garbage collector mistakenly deletes the pods 11/25/22 03:33:26.958
Nov 25 03:33:56.998: FAIL: Failed to list pods: Get "https://35.247.30.25/api/v1/namespaces/gc-1123/pods": dial tcp 35.247.30.25:443: connect: connection refused

Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.glob..func13.3()
	test/e2e/apimachinery/garbage_collector.go:475 +0x4d1
[AfterEach] [sig-api-machinery] Garbage collector
  test/e2e/framework/node/init/init.go:32
Nov 25 03:33:56.998: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:33:57.037
STEP: Collecting events from namespace "gc-1123". 11/25/22 03:33:57.037
Nov 25 03:33:57.077: INFO: Unexpected error: failed to list events in namespace "gc-1123": 
    <*url.Error | 0xc0050aca20>: {
        Op: "Get",
        URL: "https://35.247.30.25/api/v1/namespaces/gc-1123/events",
        Err: <*net.OpError | 0xc0041f0f50>{
            Op: "dial",
            Net: "tcp",
            Source: nil,
            Addr: <*net.TCPAddr | 0xc0049c3260>{
                IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 35, 247, 30, 25],
                Port: 443,
                Zone: "",
            },
            Err: <*os.SyscallError | 0xc00500e440>{
                Syscall: "connect",
                Err: <syscall.Errno>0x6f,
            },
        },
    }
Nov 25 03:33:57.077: FAIL: failed to list events in namespace "gc-1123": Get "https://35.247.30.25/api/v1/namespaces/gc-1123/events": dial tcp 35.247.30.25:443: connect: connection refused

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/debug.dumpEventsInNamespace(0xc0051125c0, {0xc002eca7b9, 0x7})
	test/e2e/framework/debug/dump.go:44 +0x191
k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc0049d0000}, {0xc002eca7b9, 0x7})
	test/e2e/framework/debug/dump.go:62 +0x8d
k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc005112650?, {0xc002eca7b9?, 0x7fa7740?})
	test/e2e/framework/debug/init/init.go:34 +0x32
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
	test/e2e/framework/framework.go:274 +0x6d
k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0004ecff0)
	test/e2e/framework/framework.go:271 +0x179
reflect.Value.call({0x6627cc0?, 0xc004693440?, 0xc0020eefb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0xc00167f5a8?})
	/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc004693440?, 0x29449fc?}, {0xae73300?, 0xc0020eef80?, 0x2d5dcbd?})
	/usr/local/go/src/reflect/value.go:368 +0xbc
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  tear down framework | framework.go:193
STEP: Destroying namespace "gc-1123" for this suite. 11/25/22 03:33:57.077
Nov 25 03:33:57.117: FAIL: Couldn't delete ns: "gc-1123": Delete "https://35.247.30.25/api/v1/namespaces/gc-1123": dial tcp 35.247.30.25:443: connect: connection refused (&url.Error{Op:"Delete", URL:"https://35.247.30.25/api/v1/namespaces/gc-1123", Err:(*net.OpError)(0xc0050d4b40)})

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach.func1()
	test/e2e/framework/framework.go:370 +0x4fe
k8s.io/kubernetes/test/e2e/framework.(*Framework).AfterEach(0xc0004ecff0)
	test/e2e/framework/framework.go:383 +0x1ca
reflect.Value.call({0x6627cc0?, 0xc0046933c0?, 0xc0037b1fb0?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0x0?})
	/usr/local/go/src/reflect/value.go:584 +0x8c5
reflect.Value.Call({0x6627cc0?, 0xc0046933c0?, 0x0?}, {0xae73300?, 0x5?, 0xc002eca740?})
	/usr/local/go/src/reflect/value.go:368 +0xbc

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should support cascading deletion of custom resources 3m36s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sGarbage\scollector\sshould\ssupport\scascading\sdeletion\sof\scustom\sresources$'

test/e2e/apimachinery/garbage_collector.go:1013
k8s.io/kubernetes/test/e2e/apimachinery.glob..func13.9()
	test/e2e/apimachinery/garbage_collector.go:1013 +0x16ca
				[BeforeEach] [sig-api-machinery] Garbage collector
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:15:49.822
Nov 25 03:15:49.822: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename gc 11/25/22 03:15:49.823
STEP: Waiting for a default service account to be provisioned in namespace 11/25/22 03:15:49.997
STEP: Waiting for kube-root-ca.crt to be provisioned in namespace 11/25/22 03:15:50.097
[BeforeEach] [sig-api-machinery] Garbage collector
  test/e2e/framework/metrics/init/init.go:31
[It] should support cascading deletion of custom resources
  test/e2e/apimachinery/garbage_collector.go:905
Nov 25 03:15:50.188: INFO: >>> kubeConfig: /workspace/.kube/config
Nov 25 03:15:53.484: INFO: created owner resource "owner56l2s"
Nov 25 03:15:53.558: INFO: created dependent resource "dependenttmht9"
Nov 25 03:15:53.747: INFO: created canary resource "canary8pcht"
Nov 25 03:18:54.004: INFO: canary last state: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"mygroup.example.com/v1beta1", "kind":"foods7n8a", "metadata":map[string]interface {}{"creationTimestamp":"2022-11-25T03:15:53Z", "generation":1, "managedFields":[]interface {}{map[string]interface {}{"apiVersion":"mygroup.example.com/v1beta1", "fieldsType":"FieldsV1", "fieldsV1":map[string]interface {}{"f:metadata":map[string]interface {}{"f:ownerReferences":map[string]interface {}{".":map[string]interface {}{}, "k:{\"uid\":\"ee315032-be4a-4867-963f-aac149f88e74\"}":map[string]interface {}{}}}}, "manager":"e2e.test", "operation":"Update", "time":"2022-11-25T03:15:53Z"}}, "name":"dependenttmht9", "ownerReferences":[]interface {}{map[string]interface {}{"apiVersion":"mygroup.example.com/v1beta1", "kind":"foods7n8a", "name":"owner56l2s", "uid":"ee315032-be4a-4867-963f-aac149f88e74"}}, "resourceVersion":"14249", "uid":"b25fc360-3e84-415f-8665-04a9406d4e9c"}}}
Nov 25 03:18:54.005: FAIL: failed waiting for canary resource "canary8pcht" to be deleted

Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.glob..func13.9()
	test/e2e/apimachinery/garbage_collector.go:1013 +0x16ca
Nov 25 03:19:24.173: FAIL: failed to delete CustomResourceDefinition: timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/apimachinery.glob..func13.9.1()
	test/e2e/apimachinery/garbage_collector.go:922 +0xbc
panic({0x70eb7e0, 0xc000408620})
	/usr/local/go/src/runtime/panic.go:884 +0x212
k8s.io/kubernetes/test/e2e/framework.Failf({0x76ed657?, 0xb?}, {0xc0008a1e18?, 0x16?, 0xc003f0de58?})
	test/e2e/framework/log.go:49 +0x12c
k8s.io/kubernetes/test/e2e/apimachinery.glob..func13.9()
	test/e2e/apimachinery/garbage_collector.go:1013 +0x16ca
[AfterEach] [sig-api-machinery] Garbage collector
  test/e2e/framework/node/init/init.go:32
Nov 25 03:19:24.173: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  test/e2e/framework/metrics/init/init.go:33
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:19:24.256
STEP: Collecting events from namespace "gc-5947". 11/25/22 03:19:24.256
STEP: Found 0 events. 11/25/22 03:19:24.297
Nov 25 03:19:24.338: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:19:24.338: INFO: 
Nov 25 03:19:24.387: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:19:24.429: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 14600 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:08 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:08 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:19:24.429: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:19:24.475: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:19:24.527: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container kube-addon-manager ready: true, restart count 4
Nov 25 03:19:24.527: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container l7-lb-controller ready: false, restart count 6
Nov 25 03:19:24.527: INFO: metadata-proxy-v0.1-vkhjm started at 2022-11-25 03:01:26 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:19:24.527: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:19:24.527: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container etcd-container ready: true, restart count 0
Nov 25 03:19:24.527: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container etcd-container ready: true, restart count 3
Nov 25 03:19:24.527: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container kube-apiserver ready: true, restart count 1
Nov 25 03:19:24.527: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container kube-controller-manager ready: true, restart count 6
Nov 25 03:19:24.527: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container kube-scheduler ready: true, restart count 5
Nov 25 03:19:24.527: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.527: INFO: 	Container konnectivity-server-container ready: true, restart count 1
Nov 25 03:19:24.717: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 25 03:19:24.717: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:19:24.759: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 14836 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-2480":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:11:04 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {node-problem-detector Update v1 2022-11-25 03:19:16 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:19:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:16 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:15:49 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:19:24.760: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:19:24.803: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:19:24.896: INFO: metrics-server-v0.5.2-867b8754b9-mkqv8 started at 2022-11-25 03:01:55 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container metrics-server ready: false, restart count 6
Nov 25 03:19:24.896: INFO: 	Container metrics-server-nanny ready: false, restart count 7
Nov 25 03:19:24.896: INFO: hostexec-bootstrap-e2e-minion-group-2842-khxh6 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container agnhost-container ready: false, restart count 2
Nov 25 03:19:24.896: INFO: test-deployment-tw5kw-54bc444df-5lzn2 started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container httpd ready: false, restart count 2
Nov 25 03:19:24.896: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:03:12 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container csi-attacher ready: false, restart count 6
Nov 25 03:19:24.896: INFO: 	Container csi-provisioner ready: false, restart count 6
Nov 25 03:19:24.896: INFO: 	Container csi-resizer ready: false, restart count 6
Nov 25 03:19:24.896: INFO: 	Container csi-snapshotter ready: false, restart count 6
Nov 25 03:19:24.896: INFO: 	Container hostpath ready: false, restart count 6
Nov 25 03:19:24.896: INFO: 	Container liveness-probe ready: false, restart count 6
Nov 25 03:19:24.896: INFO: 	Container node-driver-registrar ready: false, restart count 6
Nov 25 03:19:24.896: INFO: kube-proxy-bootstrap-e2e-minion-group-2842 started at 2022-11-25 03:01:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container kube-proxy ready: false, restart count 6
Nov 25 03:19:24.896: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:09:21 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container csi-attacher ready: true, restart count 4
Nov 25 03:19:24.896: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 25 03:19:24.896: INFO: 	Container csi-resizer ready: true, restart count 4
Nov 25 03:19:24.896: INFO: 	Container csi-snapshotter ready: true, restart count 4
Nov 25 03:19:24.896: INFO: 	Container hostpath ready: true, restart count 4
Nov 25 03:19:24.896: INFO: 	Container liveness-probe ready: true, restart count 4
Nov 25 03:19:24.896: INFO: 	Container node-driver-registrar ready: true, restart count 4
Nov 25 03:19:24.896: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:16:00 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container csi-attacher ready: false, restart count 5
Nov 25 03:19:24.896: INFO: 	Container csi-provisioner ready: false, restart count 5
Nov 25 03:19:24.896: INFO: 	Container csi-resizer ready: false, restart count 5
Nov 25 03:19:24.896: INFO: 	Container csi-snapshotter ready: false, restart count 5
Nov 25 03:19:24.896: INFO: 	Container hostpath ready: false, restart count 5
Nov 25 03:19:24.896: INFO: 	Container liveness-probe ready: false, restart count 5
Nov 25 03:19:24.896: INFO: 	Container node-driver-registrar ready: false, restart count 5
Nov 25 03:19:24.896: INFO: sample-webhook-deployment-865554f4d9-s2mvw started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container sample-webhook ready: true, restart count 3
Nov 25 03:19:24.896: INFO: pod-server-1 started at 2022-11-25 03:15:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:19:24.896: INFO: ss2-0 started at 2022-11-25 03:15:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:19:24.896: INFO: konnectivity-agent-xr8pf started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container konnectivity-agent ready: false, restart count 6
Nov 25 03:19:24.896: INFO: netserver-0 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container webserver ready: false, restart count 4
Nov 25 03:19:24.896: INFO: metadata-proxy-v0.1-4nskz started at 2022-11-25 03:01:32 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:19:24.896: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:19:24.896: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:19:25.133: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-2842
Nov 25 03:19:25.133: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:19:25.174: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 14833 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-provisioning-1494":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-1554":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:15:58 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:19:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {node-problem-detector Update v1 2022-11-25 03:19:17 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:17 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:16:42 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-provisioning-1494^7981ca50-6c6f-11ed-97be-7e6aeec89790,DevicePath:,},},Config:nil,},}
Nov 25 03:19:25.175: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
Nov 25 03:19:25.218: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
Nov 25 03:19:25.283: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:15:52 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container csi-provisioner ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container csi-resizer ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container csi-snapshotter ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container hostpath ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container liveness-probe ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container node-driver-registrar ready: true, restart count 3
Nov 25 03:19:25.283: INFO: test-container-pod started at 2022-11-25 03:12:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container webserver ready: true, restart count 3
Nov 25 03:19:25.283: INFO: pod-subpath-test-dynamicpv-xn2k started at 2022-11-25 03:15:57 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container test-container-subpath-dynamicpv-xn2k ready: false, restart count 0
Nov 25 03:19:25.283: INFO: 	Container test-container-volume-dynamicpv-xn2k ready: false, restart count 0
Nov 25 03:19:25.283: INFO: csi-mockplugin-0 started at 2022-11-25 03:13:23 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:19:25.283: INFO: 	Container driver-registrar ready: true, restart count 1
Nov 25 03:19:25.283: INFO: 	Container mock ready: true, restart count 1
Nov 25 03:19:25.283: INFO: kube-proxy-bootstrap-e2e-minion-group-4j20 started at 2022-11-25 03:01:30 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container kube-proxy ready: false, restart count 7
Nov 25 03:19:25.283: INFO: execpodndnps started at 2022-11-25 03:15:48 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container agnhost-container ready: true, restart count 1
Nov 25 03:19:25.283: INFO: dns-test-fa248686-6c83-425f-834a-b5c24a5cd609 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container jessie-querier ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container querier ready: true, restart count 3
Nov 25 03:19:25.283: INFO: 	Container webserver ready: true, restart count 3
Nov 25 03:19:25.283: INFO: netserver-1 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container webserver ready: false, restart count 2
Nov 25 03:19:25.283: INFO: agnhost started at 2022-11-25 03:03:43 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container agnhost ready: false, restart count 0
Nov 25 03:19:25.283: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:09:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container csi-attacher ready: true, restart count 3
Nov 25 03:19:25.283: INFO: host-test-container-pod started at 2022-11-25 03:12:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 25 03:19:25.283: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:13:23 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container csi-attacher ready: false, restart count 1
Nov 25 03:19:25.283: INFO: ss2-2 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container webserver ready: true, restart count 1
Nov 25 03:19:25.283: INFO: pvc-volume-tester-5snsv started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:19:25.283: INFO: pod2 started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 25 03:19:25.283: INFO: csi-mockplugin-0 started at 2022-11-25 03:09:45 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container csi-provisioner ready: true, restart count 4
Nov 25 03:19:25.283: INFO: 	Container driver-registrar ready: true, restart count 4
Nov 25 03:19:25.283: INFO: 	Container mock ready: true, restart count 4
Nov 25 03:19:25.283: INFO: konnectivity-agent-wbnhz started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container konnectivity-agent ready: false, restart count 7
Nov 25 03:19:25.283: INFO: metadata-proxy-v0.1-s4hwh started at 2022-11-25 03:01:31 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:19:25.283: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:19:25.283: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:19:25.540: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-4j20
Nov 25 03:19:25.540: INFO: 
Logging node info for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:19:25.582: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-q7zm    129634e6-c3de-48ca-90ee-023686f517c9 14849 0 2022-11-25 03:01:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-q7zm kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-q7zm topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.0.0/24\"":{}}}} } {kubelet Update v1 2022-11-25 03:01:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:03:59 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:17:32 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status} {node-problem-detector Update v1 2022-11-25 03:19:21 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status}]},Spec:NodeSpec{PodCIDR:10.64.0.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-q7zm,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.0.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815438336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553294336 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:04:18 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:19:21 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:17:32 +0000 UTC,LastTransitionTime:2022-11-25 03:01:22 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.3,},NodeAddress{Type:ExternalIP,Address:34.82.45.126,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-q7zm.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:e2712a1c866a89f59e9679218a9149a1,SystemUUID:e2712a1c-866a-89f5-9e96-79218a9149a1,BootID:16a1e367-c564-4244-8449-319cd751f412,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/snapshot-controller@sha256:823c75d0c45d1427f6d850070956d9ca657140a7bbf828381541d1d808475280 registry.k8s.io/sig-storage/snapshot-controller:v6.1.0],SizeBytes:22620891,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/cpa/cluster-proportional-autoscaler@sha256:fd636b33485c7826fb20ef0688a83ee0910317dbb6c0c6f3ad14661c1db25def registry.k8s.io/cpa/cluster-proportional-autoscaler:1.8.4],SizeBytes:15209393,},ContainerImage{Names:[registry.k8s.io/coredns/coredns@sha256:8e352a029d304ca7431c6507b56800636c321cb52289686a581ab70aaa8a2e2a registry.k8s.io/coredns/coredns:v1.9.3],SizeBytes:14837849,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64@sha256:7eb7b3cee4d33c10c49893ad3c386232b86d4067de5251294d4c620d6e072b93 registry.k8s.io/networking/ingress-gce-404-server-with-metrics-amd64:v1.10.11],SizeBytes:6463068,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:19:25.583: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-q7zm
Nov 25 03:19:25.626: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-q7zm
Nov 25 03:19:25.683: INFO: pvc-volume-tester-s584h started at 2022-11-25 03:15:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:19:25.683: INFO: volume-snapshot-controller-0 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container volume-snapshot-controller ready: false, restart count 6
Nov 25 03:19:25.683: INFO: konnectivity-agent-gd8tk started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container konnectivity-agent ready: true, restart count 5
Nov 25 03:19:25.683: INFO: kube-proxy-bootstrap-e2e-minion-group-q7zm started at 2022-11-25 03:01:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container kube-proxy ready: false, restart count 7
Nov 25 03:19:25.683: INFO: csi-mockplugin-0 started at 2022-11-25 03:11:56 +0000 UTC (0+4 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container busybox ready: false, restart count 4
Nov 25 03:19:25.683: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:19:25.683: INFO: 	Container driver-registrar ready: false, restart count 3
Nov 25 03:19:25.683: INFO: 	Container mock ready: false, restart count 3
Nov 25 03:19:25.683: INFO: pod-projected-secrets-fd485c27-166a-49eb-b3cc-a98f3b0470fa started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container projected-secret-volume-test ready: false, restart count 0
Nov 25 03:19:25.683: INFO: l7-default-backend-8549d69d99-k696w started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container default-http-backend ready: true, restart count 0
Nov 25 03:19:25.683: INFO: execpodlc59j started at 2022-11-25 03:13:12 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container agnhost-container ready: false, restart count 4
Nov 25 03:19:25.683: INFO: netserver-2 started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:19:25.683: INFO: downwardapi-volume-927901cf-394f-4333-8c62-7d60bf4ee648 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:19:25.683: INFO: kube-dns-autoscaler-5f6455f985-77mxq started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container autoscaler ready: false, restart count 7
Nov 25 03:19:25.683: INFO: pod-secrets-0a07040a-2718-4694-83b9-20611c00d694 started at 2022-11-25 03:11:56 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container creates-volume-test ready: false, restart count 0
Nov 25 03:19:25.683: INFO: 	Container dels-volume-test ready: false, restart count 0
Nov 25 03:19:25.683: INFO: 	Container upds-volume-test ready: false, restart count 0
Nov 25 03:19:25.683: INFO: pod-handle-http-request started at 2022-11-25 03:15:49 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container container-handle-http-request ready: true, restart count 1
Nov 25 03:19:25.683: INFO: 	Container container-handle-https-request ready: true, restart count 1
Nov 25 03:19:25.683: INFO: pod1 started at 2022-11-25 03:13:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:19:25.683: INFO: metadata-proxy-v0.1-pcvjx started at 2022-11-25 03:01:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:19:25.683: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:19:25.683: INFO: coredns-6d97d5ddb-ts7gx started at 2022-11-25 03:01:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container coredns ready: false, restart count 7
Nov 25 03:19:25.683: INFO: coredns-6d97d5ddb-r9926 started at 2022-11-25 03:01:36 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container coredns ready: false, restart count 7
Nov 25 03:19:25.683: INFO: ss2-1 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container webserver ready: true, restart count 0
Nov 25 03:19:25.683: INFO: nfs-server started at 2022-11-25 03:09:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container nfs-server ready: false, restart count 6
Nov 25 03:19:25.683: INFO: recycler-for-nfs-h8mdt started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:19:25.683: INFO: 	Container pv-recycler ready: false, restart count 0
Nov 25 03:19:25.902: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-q7zm
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  tear down framework | framework.go:193
STEP: Destroying namespace "gc-5947" for this suite. 11/25/22 03:19:25.902

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] Garbage collector should support orphan deletion of custom resources 2m1s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sGarbage\scollector\sshould\ssupport\sorphan\sdeletion\sof\scustom\sresources$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010acff0)
	test/e2e/framework/framework.go:241 +0x96f
There were additional failures detected after the initial failure:
[INTERRUPTED]
Interrupted by User
In [DeferCleanup (Each)] at: dump namespaces | framework.go:196

This is the Progress Report generated when the interrupt was received:
  [sig-api-machinery] Garbage collector should support orphan deletion of custom resources (Spec Runtime: 2m1.374s)
    test/e2e/apimachinery/garbage_collector.go:1040
    In [DeferCleanup (Each)] (Node Runtime: 1.281s)
      dump namespaces | framework.go:196
      At [By Step] Found 0 events. (Step Runtime: 1.237s)
        test/e2e/framework/debug/dump.go:46

      Spec Goroutine
      goroutine 8327 [select]
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*ClientConn).RoundTrip(0xc000e81800, 0xc001ca6900)
          vendor/golang.org/x/net/http2/transport.go:1200
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*Transport).RoundTripOpt(0xc000399b00, 0xc001ca6900, {0xe0?})
          vendor/golang.org/x/net/http2/transport.go:519
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*Transport).RoundTrip(...)
          vendor/golang.org/x/net/http2/transport.go:480
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip({0xc003e8a000?}, 0xc001ca6900?)
          vendor/golang.org/x/net/http2/transport.go:3020
        net/http.(*Transport).roundTrip(0xc003e8a000, 0xc001ca6900)
          /usr/local/go/src/net/http/transport.go:540
        net/http.(*Transport).RoundTrip(0x6fe4b20?, 0xc003a09170?)
          /usr/local/go/src/net/http/roundtrip.go:17
        k8s.io/kubernetes/vendor/k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip(0xc004cb1b60, 0xc001ca6800)
          vendor/k8s.io/client-go/transport/round_trippers.go:317
        k8s.io/kubernetes/vendor/k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip(0xc004789480, 0xc001ca6700)
          vendor/k8s.io/client-go/transport/round_trippers.go:168
        net/http.send(0xc001ca6700, {0x7fad100, 0xc004789480}, {0x74d54e0?, 0x1?, 0x0?})
          /usr/local/go/src/net/http/client.go:251
        net/http.(*Client).send(0xc004cb1b90, 0xc001ca6700, {0x7f607f7eca68?, 0x100?, 0x0?})
          /usr/local/go/src/net/http/client.go:175
        net/http.(*Client).do(0xc004cb1b90, 0xc001ca6700)
          /usr/local/go/src/net/http/client.go:715
        net/http.(*Client).Do(...)
          /usr/local/go/src/net/http/client.go:581
        k8s.io/kubernetes/vendor/k8s.io/client-go/rest.(*Request).request(0xc001ca6500, {0x7fe0bc8, 0xc0000820e0}, 0x0?)
          vendor/k8s.io/client-go/rest/request.go:964
        k8s.io/kubernetes/vendor/k8s.io/client-go/rest.(*Request).Do(0xc001ca6500, {0x7fe0bc8, 0xc0000820e0})
          vendor/k8s.io/client-go/rest/request.go:1005
        k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes/typed/core/v1.(*events).List(0xc0059c2ea0, {0x7fe0bc8, 0xc0000820e0}, {{{0x0, 0x0}, {0x0, 0x0}}, {0x0, 0x0}, {0xc001622c80, ...}, ...})
          vendor/k8s.io/client-go/kubernetes/typed/core/v1/event.go:95
      > k8s.io/kubernetes/test/e2e/framework/debug.getNodeEvents({0x801de88, 0xc002a91a00}, {0xc004d517c0, 0x1f})
          test/e2e/framework/debug/dump.go:181
      > k8s.io/kubernetes/test/e2e/framework/debug.DumpNodeDebugInfo({0x801de88, 0xc002a91a00}, {0xc0055e46c0?, 0x4, 0xc8?}, 0x78959b8)
          test/e2e/framework/debug/dump.go:117
      > k8s.io/kubernetes/test/e2e/framework/debug.dumpAllNodeInfo({0x801de88, 0xc002a91a00}, 0xc000d7db90)
          test/e2e/framework/debug/dump.go:103
      > k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc002a91a00}, {0xc0030d9440, 0x6})
          test/e2e/framework/debug/dump.go:79
      > k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc000942650?, {0xc0030d9440?, 0x7fa7740?})
          test/e2e/framework/debug/init/init.go:34
      > k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
          test/e2e/framework/framework.go:274
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2.By({0x7696cac, 0x28}, {0xc001ad8c70, 0x1, 0x1?})
          vendor/github.com/onsi/ginkgo/v2/core_dsl.go:535
      > k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0010acff0)
          test/e2e/framework/framework.go:271
        reflect.Value.call({0x6627cc0?, 0xc0049f6f10?, 0xc0000caf08?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0xc000b1c930?})
          /usr/local/go/src/reflect/value.go:584
        reflect.Value.Call({0x6627cc0?, 0xc0049f6f10?, 0x0?}, {0xae73300?, 0x0?, 0x0?})
          /usr/local/go/src/reflect/value.go:368
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.NewCleanupNode.func3()
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:571
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000b1c900})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738

      Other Goroutines
      goroutine 107 [running]
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractRunningGoroutines()
          vendor/github.com/onsi/ginkgo/v2/internal/progress_report.go:184
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.NewProgressReport(_, {{0xc0049f6e30, 0x1, 0x1}, {0xc0046c8d00, 0x1, 0x1}, {0xc0017563d8, 0x1, 0x1}, ...}, ...)
          vendor/github.com/onsi/ginkgo/v2/internal/progress_report.go:78
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).generateProgressReport(_, _)
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:286
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode(_, {0x3d98, 0x20000, {0x0, 0x0}, 0xc0049f6f80, {{0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:821
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*group).attemptSpec(0xc0046c8cc0, 0x1, {{0xc001c8a000?, 0xc0046c8d00?, 0x1?}, 0x0?})
          vendor/github.com/onsi/ginkgo/v2/internal/group.go:271
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*group).run(0xc0046c8cc0, {0xc0010c9c80?, 0x1?, 0x0?})
          vendor/github.com/onsi/ginkgo/v2/internal/group.go:329
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runSpecs(0xc00012e000, {0x75fe576, 0x14}, {0xae73300, 0x0, 0x0}, {0xc0000884b4, 0x4c}, 0x0, {0xc003d80000, ...})
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:388
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).Run(_, {_, _}, {_, _, _}, {_, _}, _, {0x7fe6740, ...}, ...)
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:109
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2.RunSpecs({0x7fae220, 0xc000220ea0}, {0x75fe576, 0x14}, {0xc0008538b8?, 0x2, 0x0?})
          vendor/github.com/onsi/ginkgo/v2/core_dsl.go:297
        k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000220ea0?)
          test/e2e/e2e.go:127
        k8s.io/kubernetes/test/e2e.TestE2E(0x0?)
          test/e2e/e2e_test.go:146
        testing.tRunner(0xc000220ea0, 0x78952c8)
          /usr/local/go/src/testing/testing.go:1446
        testing.(*T).Run
          /usr/local/go/src/testing/testing.go:1493

      goroutine 1 [chan receive, 43 minutes]
        testing.(*T).Run(0xc000220d00, {0x75bea0a?, 0x2755a25?}, 0x78952c8)
          /usr/local/go/src/testing/testing.go:1494
        testing.runTests.func1(0xc000220d00?)
          /usr/local/go/src/testing/testing.go:1846
        testing.tRunner(0xc000220d00, 0xc00103f7f0)
          /usr/local/go/src/testing/testing.go:1446
        testing.runTests(0xc000f345a0?, {0xac3b7e0, 0x1, 0x1}, {0x0?, 0x39c0754749e178f3?, 0x0?})
          /usr/local/go/src/testing/testing.go:1844
        testing.(*M).Run(0xc000f345a0)
          /usr/local/go/src/testing/testing.go:1726
        k8s.io/kubernetes/test/e2e.TestMain(0xffffffffffffffff?)
          test/e2e/e2e_test.go:142
        main.main()
          /tmp.k8s/go-build3837713645/b001/_testmain.go:49

      goroutine 108 [select, 2 minutes]
        k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*flushDaemon).run.func1()
          vendor/k8s.io/klog/v2/klog.go:1135
        k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*flushDaemon).run
          vendor/k8s.io/klog/v2/klog.go:1131

      goroutine 109 [IO wait]
        internal/poll.runtime_pollWait(0x7f60569d79f8, 0x72)
          /usr/local/go/src/runtime/netpoll.go:305
        internal/poll.(*pollDesc).wait(0xc000399d00?, 0xc000e9c000?, 0x0)
          /usr/local/go/src/internal/poll/fd_poll_runtime.go:84
        internal/poll.(*pollDesc).waitRead(...)
          /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).Read(0xc000399d00, {0xc000e9c000, 0x1000, 0x1000})
          /usr/local/go/src/internal/poll/fd_unix.go:167
        net.(*netFD).Read(0xc000399d00, {0xc000e9c000?, 0xc0038d0a2b?, 0x16?})
          /usr/local/go/src/net/fd_posix.go:55
        net.(*conn).Read(0xc000b2f938, {0xc000e9c000?, 0xc0038d0a2b?, 0x66795c0?})
          /usr/local/go/src/net/net.go:183
        bufio.(*Reader).Read(0xc000e74ae0, {0xc000e8e090, 0x1, 0xc0000c8d70?})
          /usr/local/go/src/bufio/bufio.go:237
        io.ReadAtLeast({0x7fa3a20, 0xc000e74ae0}, {0xc000e8e090, 0x1, 0x9}, 0x1)
          /usr/local/go/src/io/io.go:332
        io.ReadFull(...)
          /usr/local/go/src/io/io.go:351
        encoding/gob.decodeUintReader({0x7fa3a20?, 0xc000e74ae0?}, {0xc000e8e090?, 0x9?, 0x9?})
          /usr/local/go/src/encoding/gob/decode.go:120
        encoding/gob.(*Decoder).recvMessage(0xc000399d80)
          /usr/local/go/src/encoding/gob/decoder.go:81
        encoding/gob.(*Decoder).decodeTypeSequence(0xc000399d80, 0x0)
          /usr/local/go/src/encoding/gob/decoder.go:146
        encoding/gob.(*Decoder).DecodeValue(0xc000399d80, {0x65c8700?, 0xc000f787e0?, 0x8?})
          /usr/local/go/src/encoding/gob/decoder.go:225
        encoding/gob.(*Decoder).Decode(0xc000399d80, {0x65c8700?, 0xc000f787e0?})
          /usr/local/go/src/encoding/gob/decoder.go:202
        net/rpc.(*gobClientCodec).ReadResponseHeader(0x6abb600?, 0x6523640?)
          /usr/local/go/src/net/rpc/client.go:228
        net/rpc.(*Client).input(0xc000e74ba0)
          /usr/local/go/src/net/rpc/client.go:109
        net/rpc.NewClientWithCodec
          /usr/local/go/src/net/rpc/client.go:206

      goroutine 70 [select, 2 minutes]
        k8s.io/kubernetes/vendor/go.opencensus.io/stats/view.(*worker).start(0xc000398200)
          vendor/go.opencensus.io/stats/view/worker.go:276
        k8s.io/kubernetes/vendor/go.opencensus.io/stats/view.init.0
          vendor/go.opencensus.io/stats/view/worker.go:34

      goroutine 139 [syscall]
        os/signal.signal_recv()
          /usr/local/go/src/runtime/sigqueue.go:152
        os/signal.loop()
          /usr/local/go/src/os/signal/signal_unix.go:23
        os/signal.Notify.func1.1
          /usr/local/go/src/os/signal/signal.go:151

      goroutine 140 [select]
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts.func1()
          vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:105
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts
          vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:102

      goroutine 141 [select]
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts.func2(0x0?)
          vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:123
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler.(*InterruptHandler).registerForInterrupts
          vendor/github.com/onsi/ginkgo/v2/internal/interrupt_handler/interrupt_handler.go:120

      goroutine 110 [select, 43 minutes]
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.RegisterForProgressSignal.func1()
          vendor/github.com/onsi/ginkgo/v2/internal/progress_report.go:32
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.RegisterForProgressSignal
          vendor/github.com/onsi/ginkgo/v2/internal/progress_report.go:30

      goroutine 111 [select, 2 minutes]
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.startPipeFactory(0xc000e78480, 0xc000746f60)
          vendor/github.com/onsi/ginkgo/v2/internal/output_interceptor.go:77
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting
          vendor/github.com/onsi/ginkgo/v2/internal/output_interceptor.go:143

      goroutine 262 [select, 43 minutes]
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc0012f2940)
          vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:127
        k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher
          vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:76

      goroutine 8300 [select]
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*clientStream).writeRequest(0xc000fc7800, 0xc001ca6900)
          vendor/golang.org/x/net/http2/transport.go:1368
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*clientStream).doRequest(0xc0037cf7a0?, 0xc000844fa0?)
          vendor/golang.org/x/net/http2/transport.go:1230
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*ClientConn).RoundTrip
          vendor/golang.org/x/net/http2/transport.go:1159

      goroutine 7383 [IO wait]
        internal/poll.runtime_pollWait(0x7f60569d7728, 0x72)
          /usr/local/go/src/runtime/netpoll.go:305
        internal/poll.(*pollDesc).wait(0xc00522ed00?, 0xc001258000?, 0x0)
          /usr/local/go/src/internal/poll/fd_poll_runtime.go:84
        internal/poll.(*pollDesc).waitRead(...)
          /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).Read(0xc00522ed00, {0xc001258000, 0xa000, 0xa000})
          /usr/local/go/src/internal/poll/fd_unix.go:167
        net.(*netFD).Read(0xc00522ed00, {0xc001258000?, 0xc000f5ee00?, 0xc001258005?})
          /usr/local/go/src/net/fd_posix.go:55
        net.(*conn).Read(0xc000e7c5b0, {0xc001258000?, 0xa000?, 0xc000f5ee00?})
          /usr/local/go/src/net/net.go:183
        crypto/tls.(*atLeastReader).Read(0xc000d5d578, {0xc001258000?, 0x0?, 0xc001ecf860?})
          /usr/local/go/src/crypto/tls/conn.go:787
        bytes.(*Buffer).ReadFrom(0xc003ee9e78, {0x7fa3c20, 0xc000d5d578})
          /usr/local/go/src/bytes/buffer.go:202
        crypto/tls.(*Conn).readFromUntil(0xc003ee9c00, {0x7fadb60?, 0xc000e7c5b0}, 0xa000?)
          /usr/local/go/src/crypto/tls/conn.go:809
        crypto/tls.(*Conn).readRecordOrCCS(0xc003ee9c00, 0x0)
          /usr/local/go/src/crypto/tls/conn.go:616
        crypto/tls.(*Conn).readRecord(...)
          /usr/local/go/src/crypto/tls/conn.go:582
        crypto/tls.(*Conn).Read(0xc003ee9c00, {0xc0034d7000, 0x1000, 0x2d52d60?})
          /usr/local/go/src/crypto/tls/conn.go:1287
        bufio.(*Reader).Read(0xc0033a1e60, {0xc000db6120, 0x9, 0x2d61845?})
          /usr/local/go/src/bufio/bufio.go:237
        io.ReadAtLeast({0x7fa3a20, 0xc0033a1e60}, {0xc000db6120, 0x9, 0x9}, 0x9)
          /usr/local/go/src/io/io.go:332
        io.ReadFull(...)
          /usr/local/go/src/io/io.go:351
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.readFrameHeader({0xc000db6120?, 0x9?, 0xc001a08db0?}, {0x7fa3a20?, 0xc0033a1e60?})
          vendor/golang.org/x/net/http2/frame.go:237
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc000db60e0)
          vendor/golang.org/x/net/http2/frame.go:498
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*clientConnReadLoop).run(0xc001ecff98)
          vendor/golang.org/x/net/http2/transport.go:2156
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*ClientConn).readLoop(0xc000e81800)
          vendor/golang.org/x/net/http2/transport.go:2051
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*Transport).newClientConn
          vendor/golang.org/x/net/http2/transport.go:748

      goroutine 8316 [select, 2 minutes]
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1()
          vendor/github.com/onsi/ginkgo/v2/internal/output_interceptor.go:163
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting
          vendor/github.com/onsi/ginkgo/v2/internal/output_interceptor.go:153

      goroutine 8291 [IO wait, 2 minutes]
        internal/poll.runtime_pollWait(0x7f60569d7638, 0x72)
          /usr/local/go/src/runtime/netpoll.go:305
        internal/poll.(*pollDesc).wait(0xc002adce40?, 0xc003702000?, 0x1)
          /usr/local/go/src/internal/poll/fd_poll_runtime.go:84
        internal/poll.(*pollDesc).waitRead(...)
          /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).Read(0xc002adce40, {0xc003702000, 0x8000, 0x8000})
          /usr/local/go/src/internal/poll/fd_unix.go:167
        os.(*File).read(...)
          /usr/local/go/src/os/file_posix.go:31
        os.(*File).Read(0xc000b2f3f8, {0xc003702000?, 0xc004a782b8?, 0x0?})
          /usr/local/go/src/os/file.go:118
        io.copyBuffer({0x7fa40c0, 0xc004a782b8}, {0x7fadfa0, 0xc000b2f3f8}, {0x0, 0x0, 0x0})
          /usr/local/go/src/io/io.go:427
        io.Copy(...)
          /usr/local/go/src/io/io.go:386
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1.1()
          vendor/github.com/onsi/ginkgo/v2/internal/output_interceptor.go:159
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*genericOutputInterceptor).ResumeIntercepting.func1
          vendor/github.com/onsi/ginkgo/v2/internal/output_interceptor.go:158

				[BeforeEach] [sig-api-machinery] Garbage collector
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:44:15.032
Nov 25 03:44:15.032: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename gc 11/25/22 03:44:15.034
Nov 25 03:46:15.079: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc00393c1e0>: {
        msg: "wait for service account \"default\" in namespace \"gc-131\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc000113cb0>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:46:15.079: FAIL: wait for service account "default" in namespace "gc-131": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc0010acff0)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] Garbage collector
  test/e2e/framework/node/init/init.go:32
Nov 25 03:46:15.079: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:46:15.125
STEP: Collecting events from namespace "gc-131". 11/25/22 03:46:15.125
STEP: Found 0 events. 11/25/22 03:46:15.169
Nov 25 03:46:15.214: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:46:15.214: INFO: 
Nov 25 03:46:15.258: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:46:15.319: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 24490 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:42:40 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:46:15.320: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:46:15.368: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:46:15.422: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container konnectivity-server-container ready: true, restart count 2
Nov 25 03:46:15.422: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container kube-scheduler ready: false, restart count 8
Nov 25 03:46:15.422: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container kube-addon-manager ready: true, restart count 5
Nov 25 03:46:15.422: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container l7-lb-controller ready: true, restart count 11
Nov 25 03:46:15.422: INFO: metadata-proxy-v0.1-vkhjm started at 2022-11-25 03:01:26 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:46:15.422: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:46:15.422: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container etcd-container ready: true, restart count 0
Nov 25 03:46:15.422: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container etcd-container ready: true, restart count 6
Nov 25 03:46:15.422: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container kube-apiserver ready: true, restart count 2
Nov 25 03:46:15.422: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.422: INFO: 	Container kube-controller-manager ready: false, restart count 9
Nov 25 03:46:15.606: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 25 03:46:15.606: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:46:15.648: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 24891 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-6050":"bootstrap-e2e-minion-group-2842","csi-hostpath-ephemeral-8195":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-4097":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-4574":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-6534":"bootstrap-e2e-minion-group-2842","csi-mock-csi-mock-volumes-3961":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:41:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:44:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:45:29 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:44:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:44:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:44:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:44:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:44:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:44:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:44:39 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8195^f46a6b65-6c72-11ed-ac0e-6ad658b1cfa8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},},Config:nil,},}
Nov 25 03:46:15.648: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:46:15.691: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:46:15.820: INFO: webserver-5b798cb9fd-mqwts started at 2022-11-25 03:40:32 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container httpd ready: true, restart count 2
Nov 25 03:46:15.821: INFO: execpod86pz7 started at 2022-11-25 03:41:04 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container agnhost-container ready: true, restart count 0
Nov 25 03:46:15.821: INFO: metrics-server-v0.5.2-867b8754b9-mkqv8 started at 2022-11-25 03:01:55 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container metrics-server ready: false, restart count 11
Nov 25 03:46:15.821: INFO: 	Container metrics-server-nanny ready: false, restart count 12
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:39:48 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 25 03:46:15.821: INFO: webserver-5b798cb9fd-wlzvs started at 2022-11-25 03:40:40 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container httpd ready: true, restart count 4
Nov 25 03:46:15.821: INFO: hostexec-bootstrap-e2e-minion-group-2842-6lzrc started at 2022-11-25 03:33:35 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container agnhost-container ready: true, restart count 6
Nov 25 03:46:15.821: INFO: hostexec-bootstrap-e2e-minion-group-2842-khxh6 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container agnhost-container ready: true, restart count 8
Nov 25 03:46:15.821: INFO: netserver-0 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container webserver ready: true, restart count 6
Nov 25 03:46:15.821: INFO: hostexec-bootstrap-e2e-minion-group-2842-l2r55 started at 2022-11-25 03:39:46 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container agnhost-container ready: true, restart count 1
Nov 25 03:46:15.821: INFO: labelsupdate9b7ad29a-3db7-47ac-aa54-7a5dc487dd31 started at 2022-11-25 03:39:57 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:40:25 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 25 03:46:15.821: INFO: boom-server started at 2022-11-25 03:33:27 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container boom-server ready: false, restart count 2
Nov 25 03:46:15.821: INFO: csi-mockplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: true, restart count 5
Nov 25 03:46:15.821: INFO: 	Container driver-registrar ready: true, restart count 5
Nov 25 03:46:15.821: INFO: 	Container mock ready: true, restart count 5
Nov 25 03:46:15.821: INFO: hostexec-bootstrap-e2e-minion-group-2842-n25j4 started at 2022-11-25 03:39:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:46:15.821: INFO: pvc-volume-tester-w5szh started at 2022-11-25 03:33:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:46:15.821: INFO: pod-projected-configmaps-63d2a6f9-5846-48d2-961d-211689b38f2c started at 2022-11-25 03:40:04 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container agnhost-container ready: false, restart count 0
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:40:50 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 25 03:46:15.821: INFO: inline-volume-tester-nz99n started at 2022-11-25 03:40:50 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-volume-tester ready: false, restart count 0
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:03:12 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: false, restart count 11
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: false, restart count 11
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: false, restart count 11
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: false, restart count 11
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: false, restart count 11
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: false, restart count 11
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: false, restart count 11
Nov 25 03:46:15.821: INFO: kube-proxy-bootstrap-e2e-minion-group-2842 started at 2022-11-25 03:01:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container kube-proxy ready: true, restart count 11
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:09:21 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: false, restart count 8
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: false, restart count 8
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: false, restart count 8
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: false, restart count 8
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: false, restart count 8
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: false, restart count 8
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: false, restart count 8
Nov 25 03:46:15.821: INFO: sample-webhook-deployment-865554f4d9-s2mvw started at 2022-11-25 03:15:53 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container sample-webhook ready: false, restart count 7
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:16:00 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: false, restart count 10
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: false, restart count 10
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: false, restart count 10
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: false, restart count 10
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: false, restart count 10
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: false, restart count 10
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: false, restart count 10
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:33:38 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: true, restart count 7
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: true, restart count 7
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: true, restart count 7
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: true, restart count 7
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: true, restart count 7
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: true, restart count 7
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: true, restart count 7
Nov 25 03:46:15.821: INFO: affinity-clusterip-qdst4 started at 2022-11-25 03:41:25 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container affinity-clusterip ready: true, restart count 2
Nov 25 03:46:15.821: INFO: konnectivity-agent-xr8pf started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container konnectivity-agent ready: false, restart count 11
Nov 25 03:46:15.821: INFO: cronjob-test-27822460-9pjlq started at 2022-11-25 03:40:00 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container test ready: false, restart count 0
Nov 25 03:46:15.821: INFO: ss2-0 started at 2022-11-25 03:15:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container webserver ready: false, restart count 9
Nov 25 03:46:15.821: INFO: annotationupdate6fa5fa29-9c0d-42ad-bef5-f67d4990e06b started at 2022-11-25 03:41:20 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:46:15.821: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: true, restart count 5
Nov 25 03:46:15.821: INFO: hostexec-bootstrap-e2e-minion-group-2842-8q8zg started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container agnhost-container ready: false, restart count 4
Nov 25 03:46:15.821: INFO: server-envvars-a7373fd6-6127-4bab-9cf2-6ec6dbc518c1 started at 2022-11-25 03:41:40 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container srv ready: true, restart count 4
Nov 25 03:46:15.821: INFO: netserver-0 started at 2022-11-25 03:41:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container webserver ready: true, restart count 1
Nov 25 03:46:15.821: INFO: metadata-proxy-v0.1-4nskz started at 2022-11-25 03:01:32 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:46:15.821: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:46:15.821: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:40:44 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container hostpath ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 25 03:46:15.821: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 25 03:46:15.821: INFO: simpletest.rc-hp82q started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container nginx ready: true, restart count 0
Nov 25 03:46:15.821: INFO: pod-failure-failjob-pm7w5 started at 2022-11-25 03:41:14 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container c ready: false, restart count 0
Nov 25 03:46:15.821: INFO: implicit-nonroot-uid started at 2022-11-25 03:41:14 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container implicit-nonroot-uid ready: false, restart count 0
Nov 25 03:46:15.821: INFO: inline-volume-tester-jwl64 started at 2022-11-25 03:39:40 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container csi-volume-tester ready: true, restart count 0
Nov 25 03:46:15.821: INFO: pod-97e144f0-d6e3-427c-8e78-a4167cf460e0 started at 2022-11-25 03:40:18 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:15.821: INFO: 	Container write-pod ready: false, restart count 0
Nov 25 03:46:16.324: INFO: 
Latency metrics for node bootstrap-e2e-minion-group-2842
Nov 25 03:46:16.324: INFO: 
Logging node info for node bootstrap-e2e-minion-group-4j20
Nov 25 03:46:16.368: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-4j20    06bd09e1-c13d-409d-8237-b06ad6854dd9 24994 0 2022-11-25 03:01:30 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-4j20 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-4j20 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-volume-expand-9814":"bootstrap-e2e-minion-group-4j20","csi-mock-csi-mock-volumes-4927":"bootstrap-e2e-minion-group-4j20"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:30 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.2.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-25 03:26:47 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {node-problem-detector Update v1 2022-11-25 03:44:40 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kubelet Update v1 2022-11-25 03:46:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.2.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-4j20,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.2.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:44:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:44:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:44:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:44:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:44:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:44:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:44:40 +0000 UTC,LastTransitionTime:2022-11-25 03:04:15 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:30 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:42:06 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.5,},NodeAddress{Type:ExternalIP,Address:34.82.1.100,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-4j20.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:45c38f81bc1600f4c26a7673fce798c2,SystemUUID:45c38f81-bc16-00f4-c26a-7673fce798c2,BootID:edd479eb-15cd-44d3-80d6-32dd7f9713f5,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/e2e-test-images/jessie-dnsutils@sha256:24aaf2626d6b27864c29de2097e8bbb840b3a414271bf7c8995e431e47d8408e registry.k8s.io/e2e-test-images/jessie-dnsutils:1.7],SizeBytes:112030336,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/volume/nfs@sha256:3bda73f2428522b0e342af80a0b9679e8594c2126f2b3cca39ed787589741b9e registry.k8s.io/e2e-test-images/volume/nfs:1.3],SizeBytes:95836203,},ContainerImage{Names:[registry.k8s.io/sig-storage/nfs-provisioner@sha256:e943bb77c7df05ebdc8c7888b2db289b13bf9f012d6a3a5a74f14d4d5743d439 registry.k8s.io/sig-storage/nfs-provisioner:v3.0.1],SizeBytes:90632047,},ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nautilus@sha256:80ba6c8c44f9623f06e868a1aa66026c8ec438ad814f9ec95e9333b415fe3550 registry.k8s.io/e2e-test-images/nautilus:1.7],SizeBytes:49641698,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/apparmor-loader@sha256:c2ba46d9cf4549528f80d4850630b712372715e0c556d35d5c3016144365d882 registry.k8s.io/e2e-test-images/apparmor-loader:1.4],SizeBytes:8695007,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/build-image/distroless-iptables@sha256:c0ddb4794bc8a3631e3373015f913c00b230ee54f87bde9ade195b3a3415c60f registry.k8s.io/build-image/distroless-iptables:v0.1.2],SizeBytes:7685585,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonewprivs@sha256:8ac1264691820febacf3aea5d152cbde6d10685731ec14966a9401c6f47a68ac registry.k8s.io/e2e-test-images/nonewprivs:1.3],SizeBytes:3263463,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-mock-csi-mock-volumes-1554^cc0d48b0-6c6e-11ed-b925-1232e8dca603,DevicePath:,},},Config:nil,},}
Nov 25 03:46:16.369: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-4j20
------------------------------
Progress Report for Ginkgo Process #4
Interrupted by User
First interrupt received; Ginkgo will run any cleanup and reporting nodes but will skip all remaining specs.  Interrupt again to skip cleanup.
Here's a current progress report:
  [sig-api-machinery] Garbage collector should support orphan deletion of custom resources (Spec Runtime: 2m1.384s)
    test/e2e/apimachinery/garbage_collector.go:1040
    In [DeferCleanup (Each)] (Node Runtime: 1.291s)
      dump namespaces | framework.go:196
      At [By Step] Found 0 events. (Step Runtime: 1.247s)
        test/e2e/framework/debug/dump.go:46

      Spec Goroutine
      goroutine 8327 [select]
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*ClientConn).RoundTrip(0xc000e81800, 0xc001ca6900)
          vendor/golang.org/x/net/http2/transport.go:1200
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*Transport).RoundTripOpt(0xc000399b00, 0xc001ca6900, {0xe0?})
          vendor/golang.org/x/net/http2/transport.go:519
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*Transport).RoundTrip(...)
          vendor/golang.org/x/net/http2/transport.go:480
        k8s.io/kubernetes/vendor/golang.org/x/net/http2.noDialH2RoundTripper.RoundTrip({0xc003e8a000?}, 0xc001ca6900?)
          vendor/golang.org/x/net/http2/transport.go:3020
        net/http.(*Transport).roundTrip(0xc003e8a000, 0xc001ca6900)
          /usr/local/go/src/net/http/transport.go:540
        net/http.(*Transport).RoundTrip(0x6fe4b20?, 0xc003a09170?)
          /usr/local/go/src/net/http/roundtrip.go:17
        k8s.io/kubernetes/vendor/k8s.io/client-go/transport.(*bearerAuthRoundTripper).RoundTrip(0xc004cb1b60, 0xc001ca6800)
          vendor/k8s.io/client-go/transport/round_trippers.go:317
        k8s.io/kubernetes/vendor/k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip(0xc004789480, 0xc001ca6700)
          vendor/k8s.io/client-go/transport/round_trippers.go:168
        net/http.send(0xc001ca6700, {0x7fad100, 0xc004789480}, {0x74d54e0?, 0x1?, 0x0?})
          /usr/local/go/src/net/http/client.go:251
        net/http.(*Client).send(0xc004cb1b90, 0xc001ca6700, {0x7f607f7eca68?, 0x100?, 0x0?})
          /usr/local/go/src/net/http/client.go:175
        net/http.(*Client).do(0xc004cb1b90, 0xc001ca6700)
          /usr/local/go/src/net/http/client.go:715
        net/http.(*Client).Do(...)
          /usr/local/go/src/net/http/client.go:581
        k8s.io/kubernetes/vendor/k8s.io/client-go/rest.(*Request).request(0xc001ca6500, {0x7fe0bc8, 0xc0000820e0}, 0x0?)
          vendor/k8s.io/client-go/rest/request.go:964
        k8s.io/kubernetes/vendor/k8s.io/client-go/rest.(*Request).Do(0xc001ca6500, {0x7fe0bc8, 0xc0000820e0})
          vendor/k8s.io/client-go/rest/request.go:1005
        k8s.io/kubernetes/vendor/k8s.io/client-go/kubernetes/typed/core/v1.(*events).List(0xc0059c2ea0, {0x7fe0bc8, 0xc0000820e0}, {{{0x0, 0x0}, {0x0, 0x0}}, {0x0, 0x0}, {0xc001622c80, ...}, ...})
          vendor/k8s.io/client-go/kubernetes/typed/core/v1/event.go:95
      > k8s.io/kubernetes/test/e2e/framework/debug.getNodeEvents({0x801de88, 0xc002a91a00}, {0xc004d517c0, 0x1f})
          test/e2e/framework/debug/dump.go:181
      > k8s.io/kubernetes/test/e2e/framework/debug.DumpNodeDebugInfo({0x801de88, 0xc002a91a00}, {0xc0055e46c0?, 0x4, 0xc8?}, 0x78959b8)
          test/e2e/framework/debug/dump.go:117
      > k8s.io/kubernetes/test/e2e/framework/debug.dumpAllNodeInfo({0x801de88, 0xc002a91a00}, 0xc000d7db90)
          test/e2e/framework/debug/dump.go:103
      > k8s.io/kubernetes/test/e2e/framework/debug.DumpAllNamespaceInfo({0x801de88, 0xc002a91a00}, {0xc0030d9440, 0x6})
          test/e2e/framework/debug/dump.go:79
      > k8s.io/kubernetes/test/e2e/framework/debug/init.init.0.func1.1(0xc000942650?, {0xc0030d9440?, 0x7fa7740?})
          test/e2e/framework/debug/init/init.go:34
      > k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo.func1()
          test/e2e/framework/framework.go:274
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2.By({0x7696cac, 0x28}, {0xc001ad8c70, 0x1, 0x1?})
          vendor/github.com/onsi/ginkgo/v2/core_dsl.go:535
      > k8s.io/kubernetes/test/e2e/framework.(*Framework).dumpNamespaceInfo(0xc0010acff0)
          test/e2e/framework/framework.go:271
        reflect.Value.call({0x6627cc0?, 0xc0049f6f10?, 0xc0000caf08?}, {0x75b6e72, 0x4}, {0xae73300, 0x0, 0xc000b1c930?})
          /usr/local/go/src/reflect/value.go:584
        reflect.Value.Call({0x6627cc0?, 0xc0049f6f10?, 0x0?}, {0xae73300?, 0x0?, 0x0?})
          /usr/local/go/src/reflect/value.go:368
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.NewCleanupNode.func3()
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:571
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func3({0x2d591ce, 0xc000b1c900})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:449
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func2()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:750
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:738
------------------------------
Nov 25 03:46:16.420: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-4j20
[DeferCleanup (Each)] [sig-api-machinery] Garbage collector
  tear down framework | framework.go:193
STEP: Destroying namespace "gc-131" for this suite. 11/25/22 03:46:16.574
Nov 25 03:46:16.616: INFO: netserver-1 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container webserver ready: false, restart count 6
Nov 25 03:46:16.617: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:09:45 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container csi-attacher ready: false, restart count 6
Nov 25 03:46:16.617: INFO: startup-4f23e49e-3296-47e8-bc27-0c3fbc51e816 started at 2022-11-25 03:40:43 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container busybox ready: true, restart count 2
Nov 25 03:46:16.617: INFO: webserver-5b798cb9fd-59br8 started at 2022-11-25 03:40:44 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container httpd ready: true, restart count 2
Nov 25 03:46:16.617: INFO: dns-test-81912d61-3c95-4924-984a-7edb0dd1ed65 started at 2022-11-25 03:39:59 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container jessie-querier ready: true, restart count 1
Nov 25 03:46:16.617: INFO: 	Container querier ready: true, restart count 1
Nov 25 03:46:16.617: INFO: 	Container webserver ready: true, restart count 1
Nov 25 03:46:16.617: INFO: netserver-1 started at 2022-11-25 03:41:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container webserver ready: true, restart count 2
Nov 25 03:46:16.617: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:13:23 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container csi-attacher ready: true, restart count 6
Nov 25 03:46:16.617: INFO: pvc-volume-tester-5snsv started at 2022-11-25 03:11:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:46:16.617: INFO: ss2-2 started at 2022-11-25 03:16:02 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container webserver ready: true, restart count 8
Nov 25 03:46:16.617: INFO: pfpod started at 2022-11-25 03:33:28 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container portforwardtester ready: false, restart count 0
Nov 25 03:46:16.617: INFO: 	Container readiness ready: false, restart count 0
Nov 25 03:46:16.617: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:39:53 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container csi-attacher ready: true, restart count 0
Nov 25 03:46:16.617: INFO: 	Container csi-provisioner ready: true, restart count 0
Nov 25 03:46:16.617: INFO: 	Container csi-resizer ready: true, restart count 0
Nov 25 03:46:16.617: INFO: 	Container csi-snapshotter ready: true, restart count 0
Nov 25 03:46:16.617: INFO: 	Container hostpath ready: true, restart count 0
Nov 25 03:46:16.617: INFO: 	Container liveness-probe ready: true, restart count 0
Nov 25 03:46:16.617: INFO: 	Container node-driver-registrar ready: true, restart count 0
Nov 25 03:46:16.617: INFO: externalname-service-hf49p started at 2022-11-25 03:40:52 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container externalname-service ready: true, restart count 3
Nov 25 03:46:16.617: INFO: pod-handle-http-request started at 2022-11-25 03:33:23 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container container-handle-http-request ready: false, restart count 6
Nov 25 03:46:16.617: INFO: 	Container container-handle-https-request ready: false, restart count 6
Nov 25 03:46:16.617: INFO: webserver-5b798cb9fd-76fhf started at 2022-11-25 03:40:32 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container httpd ready: true, restart count 0
Nov 25 03:46:16.617: INFO: csi-mockplugin-0 started at 2022-11-25 03:09:45 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container csi-provisioner ready: false, restart count 8
Nov 25 03:46:16.617: INFO: 	Container driver-registrar ready: false, restart count 8
Nov 25 03:46:16.617: INFO: 	Container mock ready: false, restart count 8
Nov 25 03:46:16.617: INFO: konnectivity-agent-wbnhz started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container konnectivity-agent ready: false, restart count 11
Nov 25 03:46:16.617: INFO: pod-logs-websocket-6f76c37f-05bb-48a9-880c-02176eea0c1f started at 2022-11-25 03:40:15 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container main ready: true, restart count 3
Nov 25 03:46:16.617: INFO: affinity-clusterip-nrqcl started at 2022-11-25 03:41:25 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container affinity-clusterip ready: true, restart count 0
Nov 25 03:46:16.617: INFO: metadata-proxy-v0.1-s4hwh started at 2022-11-25 03:01:31 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:46:16.617: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:46:16.617: INFO: simpletest.deployment-7cf4fd9d8f-ks5v6 started at 2022-11-25 03:40:39 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container nginx ready: true, restart count 3
Nov 25 03:46:16.617: INFO: cronjob-test-27822461-tmf9z started at 2022-11-25 03:41:00 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container test ready: false, restart count 0
Nov 25 03:46:16.617: INFO: pod-e66b7f89-d8d9-46f2-90a6-f9d1fc4a6523 started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container test-container ready: false, restart count 0
Nov 25 03:46:16.617: INFO: pvc-volume-tester-hk7qj started at 2022-11-25 03:33:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:46:16.617: INFO: csi-mockplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:46:16.617: INFO: 	Container driver-registrar ready: false, restart count 4
Nov 25 03:46:16.617: INFO: 	Container mock ready: false, restart count 4
Nov 25 03:46:16.617: INFO: csi-mockplugin-0 started at 2022-11-25 03:13:23 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container csi-provisioner ready: true, restart count 6
Nov 25 03:46:16.617: INFO: 	Container driver-registrar ready: true, restart count 6
Nov 25 03:46:16.617: INFO: 	Container mock ready: true, restart count 6
Nov 25 03:46:16.617: INFO: pfpod started at 2022-11-25 03:40:30 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container portforwardtester ready: false, restart count 0
Nov 25 03:46:16.617: INFO: 	Container readiness ready: false, restart count 0
Nov 25 03:46:16.617: INFO: kube-proxy-bootstrap-e2e-minion-group-4j20 started at 2022-11-25 03:01:30 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container kube-proxy ready: false, restart count 11
Nov 25 03:46:16.617: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container csi-attacher ready: false, restart count 3
Nov 25 03:46:16.617: INFO: 	Container csi-provisioner ready: false, restart count 3
Nov 25 03:46:16.617: INFO: 	Container csi-resizer ready: false, restart count 3
Nov 25 03:46:16.617: INFO: 	Container csi-snapshotter ready: false, restart count 3
Nov 25 03:46:16.617: INFO: 	Container hostpath ready: false, restart count 3
Nov 25 03:46:16.617: INFO: 	Container liveness-probe ready: false, restart count 3
Nov 25 03:46:16.617: INFO: 	Container node-driver-registrar ready: false, restart count 3
Nov 25 03:46:16.617: INFO: dns-test-7d90b96d-40b3-47e3-80ba-fbd712e2f564 started at 2022-11-25 03:40:15 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container jessie-querier ready: false, restart count 5
Nov 25 03:46:16.617: INFO: 	Container querier ready: false, restart count 5
Nov 25 03:46:16.617: INFO: 	Container webserver ready: false, restart count 5
Nov 25 03:46:16.617: INFO: hostexec-bootstrap-e2e-minion-group-4j20-ffbwm started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container agnhost-container ready: true, restart count 5
Nov 25 03:46:16.617: INFO: sample-webhook-deployment-865554f4d9-jcmkr started at 2022-11-25 03:41:30 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container sample-webhook ready: true, restart count 4
Nov 25 03:46:16.617: INFO: annotationupdateb256174a-aa90-4993-b6eb-102898f10910 started at 2022-11-25 03:33:28 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:46:16.617: INFO: pod-projected-configmaps-5a6a0953-d39e-450b-9f2b-169a0f3f1ba5 started at 2022-11-25 03:39:59 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container createcm-volume-test ready: false, restart count 0
Nov 25 03:46:16.617: INFO: 	Container delcm-volume-test ready: false, restart count 0
Nov 25 03:46:16.617: INFO: 	Container updcm-volume-test ready: false, restart count 0
Nov 25 03:46:16.617: INFO: simpletest.rc-89gpg started at 2022-11-25 03:33:22 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:46:16.617: INFO: 	Container nginx ready: true, restart count 3

				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid

Kubernetes e2e suite [It] [sig-api-machinery] ResourceQuota should create a ResourceQuota and capture the life of a replication controller. [Conformance] 2m2s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[It\]\s\[sig\-api\-machinery\]\sResourceQuota\sshould\screate\sa\sResourceQuota\sand\scapture\sthe\slife\sof\sa\sreplication\scontroller\.\s\[Conformance\]$'

test/e2e/framework/framework.go:241
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000b51680)
	test/e2e/framework/framework.go:241 +0x96f
				[BeforeEach] [sig-api-machinery] ResourceQuota
  set up framework | framework.go:178
STEP: Creating a kubernetes client 11/25/22 03:42:15.229
Nov 25 03:42:15.229: INFO: >>> kubeConfig: /workspace/.kube/config
STEP: Building a namespace api object, basename resourcequota 11/25/22 03:42:15.231
Nov 25 03:44:15.296: INFO: Unexpected error: 
    <*fmt.wrapError | 0xc0012d8000>: {
        msg: "wait for service account \"default\" in namespace \"resourcequota-8879\": timed out waiting for the condition",
        err: <*errors.errorString | 0xc00017fa50>{
            s: "timed out waiting for the condition",
        },
    }
Nov 25 03:44:15.296: FAIL: wait for service account "default" in namespace "resourcequota-8879": timed out waiting for the condition

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).BeforeEach(0xc000b51680)
	test/e2e/framework/framework.go:241 +0x96f
[AfterEach] [sig-api-machinery] ResourceQuota
  test/e2e/framework/node/init/init.go:32
Nov 25 03:44:15.297: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
[DeferCleanup (Each)] [sig-api-machinery] ResourceQuota
  dump namespaces | framework.go:196
STEP: dump namespace information after failure 11/25/22 03:44:15.365
STEP: Collecting events from namespace "resourcequota-8879". 11/25/22 03:44:15.365
STEP: Found 0 events. 11/25/22 03:44:15.407
Nov 25 03:44:15.448: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Nov 25 03:44:15.448: INFO: 
Nov 25 03:44:15.497: INFO: 
Logging node info for node bootstrap-e2e-master
Nov 25 03:44:15.540: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-master    a98366d8-2fd7-44fb-b929-1284192417e6 24490 0 2022-11-25 03:01:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-1 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-master kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-1 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{},"f:unschedulable":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.1.0/24\"":{}},"f:taints":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:36 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}}}} status} {kubelet Update v1 2022-11-25 03:42:40 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.1.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-master,Unschedulable:true,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},Taint{Key:node.kubernetes.io/unschedulable,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.64.1.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{16656896000 0} {<nil>} 16266500Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3858366464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{1 0} {<nil>} 1 DecimalSI},ephemeral-storage: {{14991206376 0} {<nil>} 14991206376 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{3596222464 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:36 +0000 UTC,LastTransitionTime:2022-11-25 03:01:36 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:42:40 +0000 UTC,LastTransitionTime:2022-11-25 03:01:26 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.2,},NodeAddress{Type:ExternalIP,Address:35.247.30.25,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-master.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1d1aa961c7c7ac37b913dddaebf97b1f,SystemUUID:1d1aa961-c7c7-ac37-b913-dddaebf97b1f,BootID:2b5cb530-e0f6-4572-aabc-4993a4a7f7c6,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-apiserver-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:135160275,},ContainerImage{Names:[registry.k8s.io/kube-controller-manager-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:124989753,},ContainerImage{Names:[registry.k8s.io/etcd@sha256:dd75ec974b0a2a6f6bb47001ba09207976e625db898d1b16735528c009cb171c registry.k8s.io/etcd:3.5.6-0],SizeBytes:102542580,},ContainerImage{Names:[registry.k8s.io/kube-scheduler-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:57660216,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64@sha256:5db27383add6d9f4ebdf0286409ac31f7f5d273690204b341a4e37998917693b gcr.io/k8s-ingress-image-push/ingress-gce-glbc-amd64:v1.20.1],SizeBytes:36598135,},ContainerImage{Names:[registry.k8s.io/addon-manager/kube-addon-manager@sha256:49cc4e6e4a3745b427ce14b0141476ab339bb65c6bc05033019e046c8727dcb0 registry.k8s.io/addon-manager/kube-addon-manager:v9.1.6],SizeBytes:30464183,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-server@sha256:2c111f004bec24888d8cfa2a812a38fb8341350abac67dcd0ac64e709dfe389c registry.k8s.io/kas-network-proxy/proxy-server:v0.0.33],SizeBytes:22020129,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Nov 25 03:44:15.541: INFO: 
Logging kubelet events for node bootstrap-e2e-master
Nov 25 03:44:15.584: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-master
Nov 25 03:44:15.642: INFO: l7-lb-controller-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container l7-lb-controller ready: true, restart count 11
Nov 25 03:44:15.642: INFO: metadata-proxy-v0.1-vkhjm started at 2022-11-25 03:01:26 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:44:15.642: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:44:15.642: INFO: etcd-server-events-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container etcd-container ready: true, restart count 0
Nov 25 03:44:15.642: INFO: etcd-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container etcd-container ready: true, restart count 6
Nov 25 03:44:15.642: INFO: kube-apiserver-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container kube-apiserver ready: true, restart count 2
Nov 25 03:44:15.642: INFO: kube-controller-manager-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container kube-controller-manager ready: false, restart count 9
Nov 25 03:44:15.642: INFO: kube-scheduler-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container kube-scheduler ready: false, restart count 8
Nov 25 03:44:15.642: INFO: kube-addon-manager-bootstrap-e2e-master started at 2022-11-25 03:00:58 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container kube-addon-manager ready: true, restart count 5
Nov 25 03:44:15.642: INFO: konnectivity-server-bootstrap-e2e-master started at 2022-11-25 03:00:42 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:15.642: INFO: 	Container konnectivity-server-container ready: true, restart count 2
Nov 25 03:44:15.884: INFO: 
Latency metrics for node bootstrap-e2e-master
Nov 25 03:44:15.884: INFO: 
Logging node info for node bootstrap-e2e-minion-group-2842
Nov 25 03:44:15.932: INFO: Node Info: &Node{ObjectMeta:{bootstrap-e2e-minion-group-2842    840362f6-98b9-48b7-8035-0d1623b189f5 24670 0 2022-11-25 03:01:31 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:n1-standard-2 beta.kubernetes.io/os:linux cloud.google.com/metadata-proxy-ready:true failure-domain.beta.kubernetes.io/region:us-west1 failure-domain.beta.kubernetes.io/zone:us-west1-b kubernetes.io/arch:amd64 kubernetes.io/hostname:bootstrap-e2e-minion-group-2842 kubernetes.io/os:linux node.kubernetes.io/instance-type:n1-standard-2 topology.hostpath.csi/node:bootstrap-e2e-minion-group-2842 topology.kubernetes.io/region:us-west1 topology.kubernetes.io/zone:us-west1-b] map[csi.volume.kubernetes.io/nodeid:{"csi-hostpath-ephemeral-8195":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-4097":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-4574":"bootstrap-e2e-minion-group-2842","csi-hostpath-provisioning-6534":"bootstrap-e2e-minion-group-2842"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-25 03:01:31 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/instance-type":{},"f:beta.kubernetes.io/os":{},"f:cloud.google.com/metadata-proxy-ready":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-25 03:01:33 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.64.3.0/24\"":{}}}} } {node-problem-detector Update v1 2022-11-25 03:39:39 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"CorruptDockerOverlay2\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentContainerdRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentDockerRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentKubeletRestart\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"FrequentUnregisterNetDevice\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"KernelDeadlock\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"ReadonlyFilesystem\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {kube-controller-manager Update v1 2022-11-25 03:41:14 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-25 03:43:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.hostpath.csi/node":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.64.3.0/24,DoNotUseExternalID:,ProviderID:gce://k8s-jkns-gce-slow-1-3/us-west1-b/bootstrap-e2e-minion-group-2842,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.64.3.0/24],},Status:NodeStatus{Capacity:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{101203873792 0} {<nil>} 98831908Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7815430144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{attachable-volumes-gce-pd: {{127 0} {<nil>} 127 DecimalSI},cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{91083486262 0} {<nil>} 91083486262 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7553286144 0} {<nil>}  BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:FrequentDockerRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentDockerRestart,Message:docker is functioning properly,},NodeCondition{Type:FrequentContainerdRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentContainerdRestart,Message:containerd is functioning properly,},NodeCondition{Type:KernelDeadlock,Status:True,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:04:13 +0000 UTC,Reason:DockerHung,Message:kernel: INFO: task docker:12345 blocked for more than 120 seconds.,},NodeCondition{Type:ReadonlyFilesystem,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:FilesystemIsNotReadOnly,Message:Filesystem is not read-only,},NodeCondition{Type:CorruptDockerOverlay2,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoCorruptDockerOverlay2,Message:docker overlay2 is functioning properly,},NodeCondition{Type:FrequentUnregisterNetDevice,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentUnregisterNetDevice,Message:node is functioning properly,},NodeCondition{Type:FrequentKubeletRestart,Status:False,LastHeartbeatTime:2022-11-25 03:39:39 +0000 UTC,LastTransitionTime:2022-11-25 03:01:35 +0000 UTC,Reason:NoFrequentKubeletRestart,Message:kubelet is functioning properly,},NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-25 03:01:47 +0000 UTC,LastTransitionTime:2022-11-25 03:01:47 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:31 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-25 03:41:52 +0000 UTC,LastTransitionTime:2022-11-25 03:01:33 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.138.0.4,},NodeAddress{Type:ExternalIP,Address:34.145.44.11,},NodeAddress{Type:InternalDNS,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},NodeAddress{Type:Hostname,Address:bootstrap-e2e-minion-group-2842.c.k8s-jkns-gce-slow-1-3.internal,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:1851ce6484b7997774c90dfda034ab5e,SystemUUID:1851ce64-84b7-9977-74c9-0dfda034ab5e,BootID:1386e37f-672f-4134-ad2a-406b3b16fbef,KernelVersion:5.10.123+,OSImage:Container-Optimized OS from Google,ContainerRuntimeVersion:containerd://1.7.0-beta.0-149-gd06318622,KubeletVersion:v1.27.0-alpha.0.46+8f2371bcceff79,KubeProxyVersion:v1.27.0-alpha.0.46+8f2371bcceff79,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[registry.k8s.io/kube-proxy-amd64:v1.27.0-alpha.0.46_8f2371bcceff79],SizeBytes:67201736,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:16bbf38c463a4223d8cfe4da12bc61010b082a79b4bb003e2d3ba3ece5dd5f9e registry.k8s.io/e2e-test-images/agnhost:2.43],SizeBytes:51706353,},ContainerImage{Names:[gke.gcr.io/prometheus-to-sd@sha256:e739643c3939ba0b161425f45a1989eedfc4a3b166db9a7100863296b4c70510 gke.gcr.io/prometheus-to-sd:v0.11.1-gke.1],SizeBytes:48742566,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:3fe7acf013d1264ffded116b80a73dc129a449b0fccdb8d21af8279f2233f36e registry.k8s.io/e2e-test-images/httpd:2.4.39-4],SizeBytes:41901587,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/httpd@sha256:148b022f5c5da426fc2f3c14b5c0867e58ef05961510c84749ac1fddcb0fef22 registry.k8s.io/e2e-test-images/httpd:2.4.38-4],SizeBytes:40764257,},ContainerImage{Names:[registry.k8s.io/metrics-server/metrics-server@sha256:6385aec64bb97040a5e692947107b81e178555c7a5b71caa90d733e4130efc10 registry.k8s.io/metrics-server/metrics-server:v0.5.2],SizeBytes:26023008,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-provisioner@sha256:ee3b525d5b89db99da3b8eb521d9cd90cb6e9ef0fbb651e98bb37be78d36b5b8 registry.k8s.io/sig-storage/csi-provisioner:v3.3.0],SizeBytes:25491225,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-resizer@sha256:425d8f1b769398127767b06ed97ce62578a3179bcb99809ce93a1649e025ffe7 registry.k8s.io/sig-storage/csi-resizer:v1.6.0],SizeBytes:24148884,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-snapshotter@sha256:291334908ddf71a4661fd7f6d9d97274de8a5378a2b6fdfeb2ce73414a34f82f registry.k8s.io/sig-storage/csi-snapshotter:v6.1.0],SizeBytes:23881995,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-attacher@sha256:9a685020911e2725ad019dbce6e4a5ab93d51e3d4557f115e64343345e05781b registry.k8s.io/sig-storage/csi-attacher:v4.0.0],SizeBytes:23847201,},ContainerImage{Names:[registry.k8s.io/sig-storage/hostpathplugin@sha256:92257881c1d6493cf18299a24af42330f891166560047902b8d431fb66b01af5 registry.k8s.io/sig-storage/hostpathplugin:v1.9.0],SizeBytes:18758628,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/regression-issue-74839@sha256:b4f1d8d61bdad84bd50442d161d5460e4019d53e989b64220fdbc62fc87d76bf registry.k8s.io/e2e-test-images/regression-issue-74839:1.2],SizeBytes:18651485,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nonroot@sha256:ee9f50b3c64b174d296d91ca9f69a914ac30e59095dfb462b2b518ad28a63655 registry.k8s.io/e2e-test-images/nonroot:1.4],SizeBytes:17747885,},ContainerImage{Names:[registry.k8s.io/autoscaling/addon-resizer@sha256:43f129b81d28f0fdd54de6d8e7eacd5728030782e03db16087fc241ad747d3d6 registry.k8s.io/autoscaling/addon-resizer:1.8.14],SizeBytes:10153852,},ContainerImage{Names:[registry.k8s.io/sig-storage/csi-node-driver-registrar@sha256:0103eee7c35e3e0b5cd8cdca9850dc71c793cdeb6669d8be7a89440da2d06ae4 registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.5.1],SizeBytes:9133109,},ContainerImage{Names:[registry.k8s.io/sig-storage/livenessprobe@sha256:933940f13b3ea0abc62e656c1aa5c5b47c04b15d71250413a6b821bd0c58b94e registry.k8s.io/sig-storage/livenessprobe:v2.7.0],SizeBytes:8688564,},ContainerImage{Names:[registry.k8s.io/kas-network-proxy/proxy-agent@sha256:48f2a4ec3e10553a81b8dd1c6fa5fe4bcc9617f78e71c1ca89c6921335e2d7da registry.k8s.io/kas-network-proxy/proxy-agent:v0.0.33],SizeBytes:8512162,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/nginx@sha256:5c99cf6a02adda929b10321dbf4ecfa00d87be9ba4fb456006237d530ab4baa1 registry.k8s.io/e2e-test-images/nginx:1.14-4],SizeBytes:6978614,},ContainerImage{Names:[registry.k8s.io/metadata-proxy@sha256:e914645f22e946bce5165737e1b244e0a296ad1f0f81a9531adc57af2780978a registry.k8s.io/metadata-proxy:v0.1.12],SizeBytes:5301657,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf registry.k8s.io/e2e-test-images/busybox:1.29-2],SizeBytes:732424,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937 registry.k8s.io/e2e-test-images/busybox:1.29-4],SizeBytes:731990,},ContainerImage{Names:[registry.k8s.io/pause@sha256:7031c1b283388d2c2e09b57badb803c05ebed362dc88d84b480cc47f72a21097 registry.k8s.io/pause:3.9],SizeBytes:321520,},ContainerImage{Names:[registry.k8s.io/pause@sha256:9001185023633d17a2f98ff69b6ff2615b8ea02a825adffa40422f51dfdcde9d registry.k8s.io/pause:3.8],SizeBytes:311286,},},VolumesInUse:[kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-8195^f46a6b65-6c72-11ed-ac0e-6ad658b1cfa8,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3403e950-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},AttachedVolume{Name:kubernetes.io/csi/csi-hostpath-ephemeral-6050^3404b2b2-6c72-11ed-a896-ceff2a8d47ff,DevicePath:,},},Config:nil,},}
Nov 25 03:44:15.932: INFO: 
Logging kubelet events for node bootstrap-e2e-minion-group-2842
Nov 25 03:44:16.088: INFO: 
Logging pods the kubelet thinks is on node bootstrap-e2e-minion-group-2842
Nov 25 03:44:16.262: INFO: affinity-clusterip-qdst4 started at 2022-11-25 03:41:25 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container affinity-clusterip ready: true, restart count 2
Nov 25 03:44:16.262: INFO: ss2-0 started at 2022-11-25 03:15:56 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container webserver ready: false, restart count 8
Nov 25 03:44:16.262: INFO: annotationupdate6fa5fa29-9c0d-42ad-bef5-f67d4990e06b started at 2022-11-25 03:41:20 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:44:16.262: INFO: konnectivity-agent-xr8pf started at 2022-11-25 03:01:47 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container konnectivity-agent ready: false, restart count 11
Nov 25 03:44:16.262: INFO: cronjob-test-27822460-9pjlq started at 2022-11-25 03:40:00 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container test ready: false, restart count 0
Nov 25 03:44:16.262: INFO: hostexec-bootstrap-e2e-minion-group-2842-8q8zg started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container agnhost-container ready: true, restart count 3
Nov 25 03:44:16.262: INFO: csi-mockplugin-attacher-0 started at 2022-11-25 03:31:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-attacher ready: true, restart count 5
Nov 25 03:44:16.262: INFO: simpletest.rc-hp82q started at 2022-11-25 03:33:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container nginx ready: true, restart count 0
Nov 25 03:44:16.262: INFO: pod-failure-failjob-pm7w5 started at 2022-11-25 03:41:14 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container c ready: false, restart count 0
Nov 25 03:44:16.262: INFO: implicit-nonroot-uid started at 2022-11-25 03:41:14 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container implicit-nonroot-uid ready: false, restart count 0
Nov 25 03:44:16.262: INFO: server-envvars-a7373fd6-6127-4bab-9cf2-6ec6dbc518c1 started at 2022-11-25 03:41:40 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container srv ready: true, restart count 3
Nov 25 03:44:16.262: INFO: netserver-0 started at 2022-11-25 03:41:24 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container webserver ready: true, restart count 1
Nov 25 03:44:16.262: INFO: metadata-proxy-v0.1-4nskz started at 2022-11-25 03:01:32 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container metadata-proxy ready: true, restart count 0
Nov 25 03:44:16.262: INFO: 	Container prometheus-to-sd-exporter ready: true, restart count 0
Nov 25 03:44:16.262: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:40:44 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container hostpath ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 25 03:44:16.262: INFO: pod-97e144f0-d6e3-427c-8e78-a4167cf460e0 started at 2022-11-25 03:40:18 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container write-pod ready: false, restart count 0
Nov 25 03:44:16.262: INFO: inline-volume-tester-jwl64 started at 2022-11-25 03:39:40 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-volume-tester ready: false, restart count 0
Nov 25 03:44:16.262: INFO: webserver-5b798cb9fd-mqwts started at 2022-11-25 03:40:32 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container httpd ready: true, restart count 2
Nov 25 03:44:16.262: INFO: webserver-5b798cb9fd-wlzvs started at 2022-11-25 03:40:40 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container httpd ready: true, restart count 4
Nov 25 03:44:16.262: INFO: execpod86pz7 started at 2022-11-25 03:41:04 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container agnhost-container ready: true, restart count 0
Nov 25 03:44:16.262: INFO: metrics-server-v0.5.2-867b8754b9-mkqv8 started at 2022-11-25 03:01:55 +0000 UTC (0+2 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container metrics-server ready: false, restart count 11
Nov 25 03:44:16.262: INFO: 	Container metrics-server-nanny ready: false, restart count 12
Nov 25 03:44:16.262: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:39:48 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-attacher ready: true, restart count 1
Nov 25 03:44:16.262: INFO: 	Container csi-provisioner ready: true, restart count 1
Nov 25 03:44:16.262: INFO: 	Container csi-resizer ready: true, restart count 1
Nov 25 03:44:16.262: INFO: 	Container csi-snapshotter ready: true, restart count 1
Nov 25 03:44:16.262: INFO: 	Container hostpath ready: true, restart count 1
Nov 25 03:44:16.262: INFO: 	Container liveness-probe ready: true, restart count 1
Nov 25 03:44:16.262: INFO: 	Container node-driver-registrar ready: true, restart count 1
Nov 25 03:44:16.262: INFO: netserver-0 started at 2022-11-25 03:33:31 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container webserver ready: true, restart count 6
Nov 25 03:44:16.262: INFO: hostexec-bootstrap-e2e-minion-group-2842-l2r55 started at 2022-11-25 03:39:46 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container agnhost-container ready: true, restart count 1
Nov 25 03:44:16.262: INFO: hostexec-bootstrap-e2e-minion-group-2842-6lzrc started at 2022-11-25 03:33:35 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container agnhost-container ready: true, restart count 6
Nov 25 03:44:16.262: INFO: hostexec-bootstrap-e2e-minion-group-2842-khxh6 started at 2022-11-25 03:15:59 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container agnhost-container ready: false, restart count 7
Nov 25 03:44:16.262: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:40:25 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container hostpath ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 25 03:44:16.262: INFO: boom-server started at 2022-11-25 03:33:27 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container boom-server ready: true, restart count 2
Nov 25 03:44:16.262: INFO: labelsupdate9b7ad29a-3db7-47ac-aa54-7a5dc487dd31 started at 2022-11-25 03:39:57 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container client-container ready: false, restart count 0
Nov 25 03:44:16.262: INFO: pvc-volume-tester-w5szh started at 2022-11-25 03:33:19 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container volume-tester ready: false, restart count 0
Nov 25 03:44:16.262: INFO: pod-projected-configmaps-63d2a6f9-5846-48d2-961d-211689b38f2c started at 2022-11-25 03:40:04 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container agnhost-container ready: false, restart count 0
Nov 25 03:44:16.262: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:40:50 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-attacher ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-provisioner ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-resizer ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container csi-snapshotter ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container hostpath ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container liveness-probe ready: true, restart count 2
Nov 25 03:44:16.262: INFO: 	Container node-driver-registrar ready: true, restart count 2
Nov 25 03:44:16.262: INFO: csi-mockplugin-0 started at 2022-11-25 03:31:24 +0000 UTC (0+3 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-provisioner ready: false, restart count 4
Nov 25 03:44:16.262: INFO: 	Container driver-registrar ready: false, restart count 4
Nov 25 03:44:16.262: INFO: 	Container mock ready: false, restart count 4
Nov 25 03:44:16.262: INFO: hostexec-bootstrap-e2e-minion-group-2842-n25j4 started at 2022-11-25 03:39:49 +0000 UTC (0+1 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container agnhost-container ready: true, restart count 2
Nov 25 03:44:16.262: INFO: csi-hostpathplugin-0 started at 2022-11-25 03:03:12 +0000 UTC (0+7 container statuses recorded)
Nov 25 03:44:16.262: INFO: 	Container csi-attacher ready: false, restart count 11
Nov 25 03:44:16.262: INFO: 	Container csi-provisioner ready: false, restart count 11
Nov 25 03:44:16.262: INFO: 	Container csi-resizer ready: false, restart count 11
Nov 25 03:44:16.262: INFO: 	Container csi-snapshotter ready: false, restart count 11
Nov 25 03:44:16.262: INFO: 	Container hostpath ready: false, restart count 11
Nov 25 03:44:16.262: INFO: 	Container liveness-probe ready: false, restart count 11
Nov 25 03:44:16.262: INFO: 	Container node-driver-registrar ready: false, resta

Result	FAILURE
Tests	324 failed / 329 succeeded
Started	2022-11-25 02:56
Elapsed	1h3m
Revision
Builder	afe28a7d-6c6c-11ed-a6da-22215f11743a
control_plane_node_os_image	cos-97-16919-103-16
infra-commit	fe6d234c6
job-version	v1.27.0-alpha.0.46+8f2371bcceff79
kubetest-version	v20221116-7c85504268
repo	github.com/containerd/containerd
repo-commit	d06318622178a2209338d7f57eb8800b9f2e72fb
repos	{u'github.com/containerd/containerd': u'main'}
revision	v1.27.0-alpha.0.46+8f2371bcceff79
worker_node_os_image	cos-97-16919-103-16