This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 3 failed / 15 succeeded
Started2023-03-22 15:12
Elapsed1h16m
Revision
Builderff2bffb1-c8c3-11ed-abe9-f26dd6892cca
infra-commitfc8d3b97b
job-versionv1.27.0-beta.0.64+3cf9f66e90d560
kubetest-versionv20230321-850d5bc856
repok8s.io/kubernetes
repo-commit3cf9f66e90d560ac080687610933c712bcf37b39
repos{u'k8s.io/kubernetes': u'master'}
revisionv1.27.0-beta.0.64+3cf9f66e90d560

Test Failures

E2eNode Suite [It] [sig-node] LocalStorageCapacityIsolationEviction [Slow] [Serial] [Disruptive] [Feature:LocalStorageCapacityIsolation][NodeFeature:Eviction] when we run containers that should cause evictions due to pod local storage violations should eventually evict all of the correct pods 4m13s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=E2eNode\sSuite\s\[It\]\s\[sig\-node\]\sLocalStorageCapacityIsolationEviction\s\[Slow\]\s\[Serial\]\s\[Disruptive\]\s\[Feature\:LocalStorageCapacityIsolation\]\[NodeFeature\:Eviction\]\swhen\swe\srun\scontainers\sthat\sshould\scause\sevictions\sdue\sto\spod\slocal\sstorage\sviolations\s\sshould\seventually\sevict\sall\sof\sthe\scorrect\spods$'
[TIMEDOUT] A suite timeout occurred
In [AfterEach] at: test/e2e_node/eviction_test.go:620 @ 03/22/23 16:28:30.197

This is the Progress Report generated when the suite timeout occurred:
  [sig-node] LocalStorageCapacityIsolationEviction [Slow] [Serial] [Disruptive] [Feature:LocalStorageCapacityIsolation][NodeFeature:Eviction] when we run containers that should cause evictions due to pod local storage violations  should eventually evict all of the correct pods (Spec Runtime: 4m8.409s)
    test/e2e_node/eviction_test.go:563
    In [AfterEach] (Node Runtime: 22.811s)
      test/e2e_node/eviction_test.go:620
      At [By Step] deleting pod: emptydir-memory-sizelimit-pod (Step Runtime: 22.795s)
        test/e2e_node/eviction_test.go:633

      Spec Goroutine
      goroutine 8463 [select]
        k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal.(*AsyncAssertion).match(0xc000973260, {0x5b93590?, 0x88468b8}, 0x1, {0x0, 0x0, 0x0})
          vendor/github.com/onsi/gomega/internal/async_assertion.go:538
        k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal.(*AsyncAssertion).Should(0xc000973260, {0x5b93590, 0x88468b8}, {0x0, 0x0, 0x0})
          vendor/github.com/onsi/gomega/internal/async_assertion.go:145
        k8s.io/kubernetes/test/e2e/framework.asyncAssertion.Should({{0x7f6da9456088, 0xc00125fcb0}, {0xc0019f7070, 0x1, 0x1}, 0x8bb2c97000, 0x77359400, 0x0}, {0x5b93590, 0x88468b8})
          test/e2e/framework/expect.go:234
        k8s.io/kubernetes/test/e2e/framework/pod.WaitForPodNotFoundInNamespace({0x7f6da9456088, 0xc00125fcb0}, {0x5beb670?, 0xc001a58b60}, {0xc000115360, 0x1d}, {0xc000e930e0, 0x1f}, 0x0?)
          test/e2e/framework/pod/wait.go:538
        k8s.io/kubernetes/test/e2e/framework/pod.(*PodClient).DeleteSync(0xc0019a81e0, {0x7f6da9456088, 0xc00125fcb0}, {0xc000115360, 0x1d}, {{{0x0, 0x0}, {0x0, 0x0}}, 0x0, ...}, ...)
          test/e2e/framework/pod/pod_client.go:184
      > k8s.io/kubernetes/test/e2e_node.runEvictionTest.func1.3({0x7f6da9456088?, 0xc00125fcb0})
          test/e2e_node/eviction_test.go:634
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x5bb2a40?, 0xc00125fcb0})
          vendor/github.com/onsi/ginkgo/v2/internal/node.go:456
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:863
        k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/v2/internal.(*Suite).runNode
          vendor/github.com/onsi/ginkgo/v2/internal/suite.go:850

      Begin Additional Progress Reports >>
        Expected
            <*v1.Pod | 0xc000e9d680>: 
                metadata:
                  creationTimestamp: "2023-03-22T16:24:57Z"
                  deletionGracePeriodSeconds: 30
                  deletionTimestamp: "2023-03-22T16:28:37Z"
                  managedFields:
                  - apiVersion: v1
                    fieldsType: FieldsV1
                    fieldsV1:
                      f:spec:
                        f:containers:
                          k:{"name":"emptydir-memory-sizelimit-container"}:
                            .: {}
                            f:command: {}
                            f:image: {}
                            f:imagePullPolicy: {}
                            f:name: {}
                            f:resources: {}
                            f:terminationMessagePath: {}
                            f:terminationMessagePolicy: {}
                            f:volumeMounts:
                              .: {}
                              k:{"mountPath":"/test-mnt"}:
                                .: {}
                                f:mountPath: {}
                                f:name: {}
                        f:dnsPolicy: {}
                        f:enableServiceLinks: {}
                        f:nodeName: {}
                        f:restartPolicy: {}
                        f:schedulerName: {}
                        f:securityContext: {}
                        f:terminationGracePeriodSeconds: {}
                        f:volumes:
                          .: {}
                          k:{"name":"test-volume"}:
                            .: {}
                            f:emptyDir:
                              .: {}
                              f:medium: {}
                              f:sizeLimit: {}
                            f:name: {}
                    manager: e2e_node.test
                    operation: Update
                    time: "2023-03-22T16:24:57Z"
                  - apiVersion: v1
                    fieldsType: FieldsV1
                    fieldsV1:
                      f:status:
                        f:conditions:
                          .: {}
                          k:{"type":"ContainersReady"}:
                            .: {}
                            f:lastProbeTime: {}
                            f:lastTransitionTime: {}
                            f:status: {}
                            f:type: {}
                          k:{"type":"Initialized"}:
                            .: {}
                            f:lastProbeTime: {}
                            f:lastTransitionTime: {}
                            f:status: {}
                            f:type: {}
                          k:{"type":"PodScheduled"}:
                            .: {}
                            f:lastProbeTime: {}
                            f:lastTransitionTime: {}
                            f:status: {}
                            f:type: {}
                          k:{"type":"Ready"}:
                            .: {}
                            f:lastProbeTime: {}
                            f:lastTransitionTime: {}
                            f:status: {}
                            f:type: {}
                        f:containerStatuses: {}
                        f:hostIP: {}
                        f:phase: {}
                        f:podIP: {}
                        f:podIPs:
                          .: {}
                          k:{"ip":"10.85.0.19"}:
                            .: {}
                            f:ip: {}
                        f:startTime: {}
                    manager: kubelet
                    operation: Update
                    subresource: status
                    time: "2023-03-22T16:24:59Z"
                  name: emptydir-memory-sizelimit-pod
                  namespace: localstorage-eviction-test-7035
                  resourceVersion: "1597"
                  uid: badf27bb-74a2-4ac6-9c2a-dd61836bf8ef
                spec:
                  containers:
                  - command:
                    - sh
                    - -c
                    - i=0; while [ $i -lt 101 ]; do dd if=/dev/urandom of=/test-mnt/file${i} bs=1048576
                      count=1 2>/dev/null; sleep .1; i=$(($i+1)); done; while true; do sleep 5; done
                    image: registry.k8s.io/e2e-test-images/busybox:1.29-4
                    imagePullPolicy: Never
                    name: emptydir-memory-sizelimit-container
                    resources: {}
                    terminationMessagePath: /dev/termination-log
                    terminationMessagePolicy: File
                    volumeMounts:
                    - mountPath: /test-mnt
                      name: test-volume
                  dnsPolicy: Default
                  enableServiceLinks: true
                  nodeName: tmp-node-e2e-a6e0710b-fedora-coreos-37-20230218-3-0-gcp-x86-64
                  preemptionPolicy: PreemptLowerPriority
                  priority: 0
                  restartPolicy: Never
                  schedulerName: default-scheduler
                  securityContext: {}
                  terminationGracePeriodSeconds: 30
                  tolerations:
                  - effect: NoExecute
                    key: node.kubernetes.io/not-ready
                    operator: Exists
                    tolerationSeconds: 300
                  - effect: NoExecute
                    key: node.kubernetes.io/unreachable
                    operator: Exists
                    tolerationSeconds: 300
                  volumes:
                  - emptyDir:
                      medium: Memory
                      sizeLimit: 100Mi
                    name: test-volume
                status:
                  conditions:
                  - lastProbeTime: null
                    lastTransitionTime: "2023-03-22T16:24:57Z"
                    status: "True"
                    type: Initialized
                  - lastProbeTime: null
                    lastTransitionTime: "2023-03-22T16:24:59Z"
                    status: "True"
                    type: Ready
                  - lastProbeTime: null
                    lastTransitionTime: "2023-03-22T16:24:59Z"
                    status: "True"
                    type: ContainersReady
                  - lastProbeTime: null
                    lastTransitionTime: "2023-03-22T16:24:57Z"
                    status: "True"
                    type: PodScheduled
                  containerStatuses:
                  - containerID: cri-o://14ce7a2992c1c46b78403808c2c2346263a331052e47e4baf461a8c21ebefa9b
                    image: registry.k8s.io/e2e-test-images/busybox:1.29-4
                    imageID: registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937
                    lastState: {}
                    name: emptydir-memory-sizelimit-container
                    ready: true
                    restartCount: 0
                    started: true
                    state:
                      running:
                        startedAt: "2023-03-22T16:24:59Z"
                  hostIP: 10.138.0.80
                  phase: Running
                  podIP: 10.85.0.19
                  podIPs:
                  - ip: 10.85.0.19
                  qosClass: BestEffort
                  startTime: "2023-03-22T16:24:57Z"
        to be nil
      << End Additional Progress Reports

      Goroutines of Interest
      goroutine 1 [chan receive, 60 minutes]
        testing.(*T).Run(0xc000220ea0, {0x52ff476?, 0x53e765?}, 0x558dd48)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/testing/testing.go:1630
        testing.runTests.func1(0x88173a0?)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/testing/testing.go:2036
        testing.tRunner(0xc000220ea0, 0xc00096fb78)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/testing/testing.go:1576
        testing.runTests(0xc0001fe0a0?, {0x8651d70, 0x1, 0x1}, {0x30?, 0xc000b93bc0?, 0x0?})
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/testing/testing.go:2034
        testing.(*M).Run(0xc0001fe0a0)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/testing/testing.go:1906
      > k8s.io/kubernetes/test/e2e_node.TestMain(0xffffffffffffffff?)
          test/e2e_node/e2e_node_suite_test.go:145
        main.main()
          /tmp/go-build247143101/b001/_testmain.go:49

      goroutine 245 [syscall, 59 minutes]
        syscall.Syscall6(0x100?, 0xc00141bcd8?, 0x6fc80d?, 0x1?, 0x52183e0?, 0xc000123e30?, 0x47edfe?)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/syscall/syscall_linux.go:91
        os.(*Process).blockUntilWaitable(0xc000c7eae0)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/os/wait_waitid.go:32
        os.(*Process).wait(0xc000c7eae0)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/os/exec_unix.go:22
        os.(*Process).Wait(...)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/os/exec.go:132
        os/exec.(*Cmd).Wait(0xc0004fe000)
          /go/src/k8s.io/kubernetes/_output/local/.gimme/versions/go1.20.2.linux.amd64/src/os/exec/exec.go:890
      > k8s.io/kubernetes/test/e2e_node/services.(*server).start.func1()
          test/e2e_node/services/server.go:166
      > k8s.io/kubernetes/test/e2e_node/services.(*server).start
          test/e2e_node/services/server.go:123

[FAILED] A suite timeout occurred and then the following failure was recorded in the timedout node before it exited:
wait for pod "emptydir-memory-sizelimit-pod" to disappear: expected pod to not be found: Context was cancelled after 22.797s.
Expected
    <*v1.Pod | 0xc000e9d680>: 
        metadata:
          creationTimestamp: "2023-03-22T16:24:57Z"
          deletionGracePeriodSeconds: 30
          deletionTimestamp: "2023-03-22T16:28:37Z"
          managedFields:
          - apiVersion: v1
            fieldsType: FieldsV1
            fieldsV1:
              f:spec:
                f:containers:
                  k:{"name":"emptydir-memory-sizelimit-container"}:
                    .: {}
                    f:command: {}
                    f:image: {}
                    f:imagePullPolicy: {}
                    f:name: {}
                    f:resources: {}
                    f:terminationMessagePath: {}
                    f:terminationMessagePolicy: {}
                    f:volumeMounts:
                      .: {}
                      k:{"mountPath":"/test-mnt"}:
                        .: {}
                        f:mountPath: {}
                        f:name: {}
                f:dnsPolicy: {}
                f:enableServiceLinks: {}
                f:nodeName: {}
                f:restartPolicy: {}
                f:schedulerName: {}
                f:securityContext: {}
                f:terminationGracePeriodSeconds: {}
                f:volumes:
                  .: {}
                  k:{"name":"test-volume"}:
                    .: {}
                    f:emptyDir:
                      .: {}
                      f:medium: {}
                      f:sizeLimit: {}
                    f:name: {}
            manager: e2e_node.test
            operation: Update
            time: "2023-03-22T16:24:57Z"
          - apiVersion: v1
            fieldsType: FieldsV1
            fieldsV1:
              f:status:
                f:conditions:
                  .: {}
                  k:{"type":"ContainersReady"}:
                    .: {}
                    f:lastProbeTime: {}
                    f:lastTransitionTime: {}
                    f:status: {}
                    f:type: {}
                  k:{"type":"Initialized"}:
                    .: {}
                    f:lastProbeTime: {}
                    f:lastTransitionTime: {}
                    f:status: {}
                    f:type: {}
                  k:{"type":"PodScheduled"}:
                    .: {}
                    f:lastProbeTime: {}
                    f:lastTransitionTime: {}
                    f:status: {}
                    f:type: {}
                  k:{"type":"Ready"}:
                    .: {}
                    f:lastProbeTime: {}
                    f:lastTransitionTime: {}
                    f:status: {}
                    f:type: {}
                f:containerStatuses: {}
                f:hostIP: {}
                f:phase: {}
                f:podIP: {}
                f:podIPs:
                  .: {}
                  k:{"ip":"10.85.0.19"}:
                    .: {}
                    f:ip: {}
                f:startTime: {}
            manager: kubelet
            operation: Update
            subresource: status
            time: "2023-03-22T16:24:59Z"
          name: emptydir-memory-sizelimit-pod
          namespace: localstorage-eviction-test-7035
          resourceVersion: "1597"
          uid: badf27bb-74a2-4ac6-9c2a-dd61836bf8ef
        spec:
          containers:
          - command:
            - sh
            - -c
            - i=0; while [ $i -lt 101 ]; do dd if=/dev/urandom of=/test-mnt/file${i} bs=1048576
              count=1 2>/dev/null; sleep .1; i=$(($i+1)); done; while true; do sleep 5; done
            image: registry.k8s.io/e2e-test-images/busybox:1.29-4
            imagePullPolicy: Never
            name: emptydir-memory-sizelimit-container
            resources: {}
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /test-mnt
              name: test-volume
          dnsPolicy: Default
          enableServiceLinks: true
          nodeName: tmp-node-e2e-a6e0710b-fedora-coreos-37-20230218-3-0-gcp-x86-64
          preemptionPolicy: PreemptLowerPriority
          priority: 0
          restartPolicy: Never
          schedulerName: default-scheduler
          securityContext: {}
          terminationGracePeriodSeconds: 30
          tolerations:
          - effect: NoExecute
            key: node.kubernetes.io/not-ready
            operator: Exists
            tolerationSeconds: 300
          - effect: NoExecute
            key: node.kubernetes.io/unreachable
            operator: Exists
            tolerationSeconds: 300
          volumes:
          - emptyDir:
              medium: Memory
              sizeLimit: 100Mi
            name: test-volume
        status:
          conditions:
          - lastProbeTime: null
            lastTransitionTime: "2023-03-22T16:24:57Z"
            status: "True"
            type: Initialized
          - lastProbeTime: null
            lastTransitionTime: "2023-03-22T16:24:59Z"
            status: "True"
            type: Ready
          - lastProbeTime: null
            lastTransitionTime: "2023-03-22T16:24:59Z"
            status: "True"
            type: ContainersReady
          - lastProbeTime: null
            lastTransitionTime: "2023-03-22T16:24:57Z"
            status: "True"
            type: PodScheduled
          containerStatuses:
          - containerID: cri-o://14ce7a2992c1c46b78403808c2c2346263a331052e47e4baf461a8c21ebefa9b
            image: registry.k8s.io/e2e-test-images/busybox:1.29-4
            imageID: registry.k8s.io/e2e-test-images/busybox@sha256:2e0f836850e09b8b7cc937681d6194537a09fbd5f6b9e08f4d646a85128e8937
            lastState: {}
            name: emptydir-memory-sizelimit-container
            ready: true
            restartCount: 0
            started: true
            state:
              running:
                startedAt: "2023-03-22T16:24:59Z"
          hostIP: 10.138.0.80
          phase: Running
          podIP: 10.85.0.19
          podIPs:
          - ip: 10.85.0.19
          qosClass: BestEffort
          startTime: "2023-03-22T16:24:57Z"
to be nil
In [AfterEach] at: test/e2e/framework/pod/pod_client.go:184 @ 03/22/23 16:28:30.207

				
				Click to see stdout/stderrfrom junit_fedora01.xml

Find local mentions in log files | View test history on testgrid

E2eNode Suite [It] [sig-node] PriorityPidEvictionOrdering [Slow] [Serial] [Disruptive][NodeFeature:Eviction] when we run containers that should cause PIDPressure should eventually evict all of the correct pods 3m20s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=E2eNode\sSuite\s\[It\]\s\[sig\-node\]\sPriorityPidEvictionOrdering\s\[Slow\]\s\[Serial\]\s\[Disruptive\]\[NodeFeature\:Eviction\]\swhen\swe\srun\scontainers\sthat\sshould\scause\sPIDPressure\s\sshould\seventually\sevict\sall\sof\sthe\scorrect\spods$'
[FAILED] Timed out after 120.001s.
Expected
    <*errors.errorString | 0xc0014eaea0>: 
    NodeCondition: PIDPressure not encountered
    {
        s: "NodeCondition: PIDPressure not encountered",
    }
to be nil
In [It] at: test/e2e_node/eviction_test.go:571 @ 03/22/23 15:33:04.791

				
				Click to see stdout/stderrfrom junit_fedora01.xml

Filter through log files | View test history on testgrid

kubetest Node Tests 1h15m

error during go run /go/src/k8s.io/kubernetes/test/e2e_node/runner/remote/run_remote.go --cleanup -vmodule=*=4 --ssh-env=gce --results-dir=/workspace/_artifacts --project=k8s-jkns-gke-ubuntu-1-6 --zone=us-west1-b --ssh-user=core --ssh-key=/workspace/.ssh/google_compute_engine --ginkgo-flags=--nodes=1 --focus="\[NodeFeature:Eviction\]" --test_args=--container-runtime-endpoint=unix:///var/run/crio/crio.sock --container-runtime-process-name=/usr/local/bin/crio --container-runtime-pid-file= --kubelet-flags="--cgroup-driver=systemd --cgroups-per-qos=true --cgroup-root=/ --runtime-cgroups=/system.slice/crio.service --kubelet-cgroups=/system.slice/kubelet.service" --extra-log="{\"name\": \"crio.log\", \"journalctl\": [\"-u\", \"crio\"]}" --test-timeout=5h0m0s --image-config-file=/workspace/test-infra/jobs/e2e_node/crio/latest/image-config-cgrpv1.yaml: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid

Show 15 Passed Tests
Show 400 Skipped Tests