This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2020-02-05 01:18
Elapsed35m25s
Revisionmaster
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/489b707c-9d1e-4e52-b641-eab8ee74bb98/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/489b707c-9d1e-4e52-b641-eab8ee74bb98/targets/test

No Test Failures!


Error lines from build-log.txt

... skipping 42 lines ...
Kubernetes cluster:
NAME                              STATUS     ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE       KERNEL-VERSION    CONTAINER-RUNTIME
ingress-nginx-dev-control-plane   Ready      master   48s   v1.17.0   172.17.0.2    <none>        Ubuntu 19.10   4.15.0-1044-gke   containerd://1.3.2
ingress-nginx-dev-worker          NotReady   <none>   10s   v1.17.0   172.17.0.3    <none>        Ubuntu 19.10   4.15.0-1044-gke   containerd://1.3.2
ingress-nginx-dev-worker2         NotReady   <none>   11s   v1.17.0   172.17.0.4    <none>        Ubuntu 19.10   4.15.0-1044-gke   containerd://1.3.2
[dev-env] building container
Error: No such image: ingress-controller/fastcgi-helloserver:dev
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 723B 0.0s done
#1 DONE 0.1s

#2 [internal] load .dockerignore
#2 transferring context: 2B 0.0s done
... skipping 483 lines ...
k8s.io/ingress-nginx/internal/ingress/status
k8s.io/ingress-nginx/internal/ingress/controller
k8s.io/ingress-nginx/cmd/nginx
k8s.io/ingress-nginx/vendor/github.com/spf13/cobra
k8s.io/ingress-nginx/cmd/dbg
k8s.io/ingress-nginx/cmd/waitshutdown
Error: No such image: quay.io/kubernetes-ingress-controller/nginx-amd64:dev
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 2.19kB done
#2 DONE 0.0s

#1 [internal] load .dockerignore
#1 transferring context: 2B done
... skipping 473 lines ...
#6 55.63 Setting up libtinfo6:amd64 (6.1+20181013-2+deb10u2) ...
#6 56.17 Selecting previously unselected package bash.
#6 56.17 (Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 3900 files and directories currently installed.)
#6 56.26 Preparing to unpack .../archives/bash_5.0-4_amd64.deb ...
#6 56.40 Unpacking bash (5.0-4) ...
#6 57.57 Setting up bash (5.0-4) ...
#6 57.82 update-alternatives: error: alternative path /usr/share/man/man7/bash-builtins.7.gz doesn't exist

#6 58.93 Selecting previously unselected package perl-modules-5.28.
#6 58.93 (Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 3972 files and directories currently installed.)
#6 58.97 Preparing to unpack .../0-perl-modules-5.28_5.28.1-6_all.deb ...
#6 59.01 Unpacking perl-modules-5.28 (5.28.1-6) ...
#6 64.61 Selecting previously unselected package libgdbm6:amd64.
#6 64.61 Preparing to unpack .../1-libgdbm6_1.18.1-4_amd64.deb ...
... skipping 655 lines ...
STEP: Creating a kubernetes client
STEP: Building a namespace api object
Feb  5 01:28:17.822: INFO: Created namespace: e2e-tests-default-ssl-certificate-1580866097785826728-h2x92
STEP: Starting new ingress controller
[BeforeEach] [ingress-nginx] default-ssl-certificate
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/default_ssl_certificate.go:39
Feb  5 01:28:57.868: INFO: Unexpected TLS error: x509: certificate is valid for ingress.local, not *
Feb  5 01:29:00.867: INFO: Unexpected TLS error: x509: certificate is valid for ingress.local, not *
Feb  5 01:29:03.871: INFO: Unexpected TLS error: x509: certificate is valid for ingress.local, not *
[It] uses default ssl certificate for catch-all ingress
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/default_ssl_certificate.go:64
STEP: making sure new ingress is deployed
STEP: making sure new ingress is responding
STEP: making sure the configured default ssl certificate is being used
[AfterEach] [ingress-nginx] default-ssl-certificate
... skipping 518 lines ...
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/global_external_auth.go:48
[BeforeEach] when global external authentication is configured
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/global_external_auth.go:58
STEP: Adding an ingress rule for /foo
STEP: Adding an ingress rule for /bar
STEP: Adding a global-auth-url to configMap
[It] should add custom error page when global-auth-signin url is configured
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/global_external_auth.go:209
STEP: Adding a global-auth-signin to configMap
[AfterEach] [ingress-nginx] Global External Auth
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
STEP: Waiting for test namespace to no longer exist
[AfterEach] [ingress-nginx] Global External Auth
... skipping 2 lines ...

• [SLOW TEST:117.737 seconds]
[ingress-nginx] Global External Auth
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:146
  when global external authentication is configured
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/global_external_auth.go:56
    should add custom error page when global-auth-signin url is configured
    /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/global_external_auth.go:209
------------------------------
[BeforeEach] [ingress-nginx] Listen on nondefault ports
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:76
STEP: Creating a kubernetes client
STEP: Building a namespace api object
... skipping 183 lines ...
[BeforeEach] [ingress-nginx] Annotations - custom-http-errors
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/customhttperrors.go:37
[It] configures Nginx correctly
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/customhttperrors.go:44
STEP: turning on proxy_intercept_errors directive
STEP: configuring error_page directive
STEP: creating error locations
STEP: updating configuration when only custom-http-error value changes
STEP: ignoring duplicate values (503 in this case) per server
STEP: using the custom default-backend from annotation for upstream
[AfterEach] [ingress-nginx] Annotations - custom-http-errors
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
STEP: Waiting for test namespace to no longer exist
[AfterEach] [ingress-nginx] Annotations - custom-http-errors
... skipping 560 lines ...
STEP: Creating a kubernetes client
STEP: Building a namespace api object
Feb  5 01:33:59.556: INFO: Created namespace: e2e-tests-ewma-1580866439533163175-tth4d
STEP: Starting new ingress controller
[BeforeEach] [ingress-nginx] Load Balance - EWMA
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/loadbalance/ewma.go:35
[It] does not fail requests
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/loadbalance/ewma.go:41
Feb  5 01:36:19.701: INFO: Request distribution: map[echo-69d6c66768-hx5dj:17 echo-69d6c66768-nxshz:5 echo-69d6c66768-t2pbp:8]
[AfterEach] [ingress-nginx] Load Balance - EWMA
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
STEP: Waiting for test namespace to no longer exist


• [SLOW TEST:140.366 seconds]
[ingress-nginx] Load Balance - EWMA
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:146
  does not fail requests
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/loadbalance/ewma.go:41
------------------------------
[BeforeEach] [ingress-nginx] Annotations - Auth
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:76
STEP: Creating a kubernetes client
STEP: Building a namespace api object
... skipping 7 lines ...
STEP: Adding an ingress rule for /bar
STEP: Adding an ingress rule for /foo
STEP: Adding an ingress rule for /bar
[It] should deny login for different servers
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/auth.go:513
STEP: logging into server thisHost /foo
STEP: receiving an internal server error without cache on thisHost location /bar
[AfterEach] [ingress-nginx] Annotations - Auth
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
STEP: Waiting for test namespace to no longer exist
[AfterEach] [ingress-nginx] Annotations - Auth
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/auth.go:43

... skipping 452 lines ...
• Failure in Spec Setup (BeforeEach) [607.512 seconds]
[ingress-nginx] X-Forwarded headers
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:146
  should trust X-Forwarded headers when setting is true [BeforeEach]
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/forwarded_headers.go:44

  Unexpected error:
      <*errors.errorString | 0xc00043a4c0>: {
          s: "unexpected error waiting for ingress controller deployment: exit status 1.\nLogs:\ndeploying NGINX Ingress controller in namespace e2e-tests-forwarded-headers-1580866097949249428-p4w4m\nserviceaccount/nginx-ingress-serviceaccount created\nrole.rbac.authorization.k8s.io/nginx-ingress-role created\nclusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole-e2e-tests-forwarded-headers-1580866097949249428-p4w4m created\nrolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created\nclusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding-e2e-tests-forwarded-headers-1580866097949249428-p4w4m created\nconfigmap/nginx-configuration created\nconfigmap/tcp-services created\nconfigmap/udp-services created\nservice/ingress-nginx created\ndeployment.apps/nginx-ingress-controller created\nlimitrange/ingress-nginx created\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nW0205 01:31:23.603407     542 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server (\"unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)\") has prevented the request from succeeding\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nW0205 01:34:24.732793     542 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server (\"unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)\") has prevented the request from succeeding\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nW0205 01:37:25.989141     542 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server (\"unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)\") has prevented the request from succeeding\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nerror: deployment \"nginx-ingress-controller\" exceeded its progress deadline\nObtaining ingress controller pod logs...\nI0205 01:35:08.519021       8 leaderelection.go:252] successfully acquired lease e2e-tests-forwarded-headers-1580866097949249428-p4w4m/ingress-controller-leader-nginx\nI0205 01:35:08.520238       8 status.go:86] new leader elected: nginx-ingress-controller-66774d4dc-98x2v\nI0205 01:35:11.744483       8 main.go:152] Received SIGTERM, shutting down\nI0205 01:35:11.891649       8 nginx.go:391] Shutting down controller queues\nI0205 01:35:11.892153       8 status.go:117] updating status of Ingress rules (remove)\nI0205 01:35:12.166501       8 status.go:136] removing address from ingress status ([10.96.138.250])\nI0205 01:35:12.166838       8 nginx.go:407] Stopping NGINX process\nE0205 01:35:12.170095       8 controller.go:149] Unexpected failure reloading the backend:\nsignal: terminated\nW0205 01:35:12.170126       8 queue.go:130] requeuing initial-sync, err signal: terminated\n",
      }
      unexpected error waiting for ingress controller deployment: exit status 1.
      Logs:
      deploying NGINX Ingress controller in namespace e2e-tests-forwarded-headers-1580866097949249428-p4w4m
      serviceaccount/nginx-ingress-serviceaccount created
      role.rbac.authorization.k8s.io/nginx-ingress-role created
      clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole-e2e-tests-forwarded-headers-1580866097949249428-p4w4m created
      rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
... skipping 2 lines ...
      configmap/tcp-services created
      configmap/udp-services created
      service/ingress-nginx created
      deployment.apps/nginx-ingress-controller created
      limitrange/ingress-nginx created
      Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
      W0205 01:31:23.603407     542 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server ("unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)") has prevented the request from succeeding
      Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
      W0205 01:34:24.732793     542 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server ("unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)") has prevented the request from succeeding
      Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
      W0205 01:37:25.989141     542 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server ("unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)") has prevented the request from succeeding
      Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
      error: deployment "nginx-ingress-controller" exceeded its progress deadline
      Obtaining ingress controller pod logs...
      I0205 01:35:08.519021       8 leaderelection.go:252] successfully acquired lease e2e-tests-forwarded-headers-1580866097949249428-p4w4m/ingress-controller-leader-nginx
      I0205 01:35:08.520238       8 status.go:86] new leader elected: nginx-ingress-controller-66774d4dc-98x2v
      I0205 01:35:11.744483       8 main.go:152] Received SIGTERM, shutting down
      I0205 01:35:11.891649       8 nginx.go:391] Shutting down controller queues
      I0205 01:35:11.892153       8 status.go:117] updating status of Ingress rules (remove)
... skipping 974 lines ...
		
		-- init modules
		local ok, res
		
		ok, res = pcall(require, "lua_ingress")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		lua_ingress = res
		lua_ingress.set_config({
			use_forwarded_headers = false,
			is_ssl_passthrough_enabled = false,
			http_redirect_code = 308,
... skipping 5 lines ...
			hsts_preload = false,
		})
		end
		
		ok, res = pcall(require, "configuration")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		configuration = res
		end
		
		ok, res = pcall(require, "balancer")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		balancer = res
		end
		
		ok, res = pcall(require, "monitor")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		monitor = res
		end
		
		ok, res = pcall(require, "certificate")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		certificate = res
		end
		
		ok, res = pcall(require, "plugins")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		plugins = res
		end
		-- load all plugins that'll be used here
	plugins.init({})
	}
... skipping 87 lines ...
		
		default 1;
	}
	
	access_log /var/log/nginx/access.log upstreaminfo  if=$loggable;
	
	error_log  /var/log/nginx/error.log notice;
	
	resolver 10.96.0.10 valid=30s;
	
	# See https://www.nginx.com/blog/websocket-nginx
	map $http_upgrade $connection_upgrade {
		default          upgrade;
... skipping 194 lines ...
			proxy_request_buffering                 on;
			proxy_http_version                      1.1;
			
			proxy_cookie_domain                     off;
			proxy_cookie_path                       off;
			
			# In case of errors try the next upstream server before returning an error
			proxy_next_upstream                     error timeout;
			proxy_next_upstream_timeout             0;
			proxy_next_upstream_tries               3;
			
			proxy_pass http://upstream_balancer;
			
			proxy_redirect                          off;
... skipping 4 lines ...
		location /healthz {
			
			access_log off;
			return 200;
		}
		
		# this is required to avoid error if nginx is being monitored
		# with an external software (like sysdig)
		location /nginx_status {
			
			allow 127.0.0.1;
			
			allow ::1;
... skipping 121 lines ...
			proxy_request_buffering                 on;
			proxy_http_version                      1.1;
			
			proxy_cookie_domain                     off;
			proxy_cookie_path                       off;
			
			# In case of errors try the next upstream server before returning an error
			proxy_next_upstream                     error timeout;
			proxy_next_upstream_timeout             0;
			proxy_next_upstream_tries               3;
			
			proxy_pass http://upstream_balancer;
			
			proxy_redirect                          off;
... skipping 76 lines ...
		
		-- init modules
		local ok, res
		
		ok, res = pcall(require, "configuration")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		configuration = res
		end
		
		ok, res = pcall(require, "tcp_udp_configuration")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		tcp_udp_configuration = res
		end
		
		ok, res = pcall(require, "tcp_udp_balancer")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		tcp_udp_balancer = res
		end
	}
	
	init_worker_by_lua_block {
... skipping 3 lines ...
	lua_add_variable $proxy_upstream_name;
	
	log_format log_stream '[$remote_addr] [$time_local] $protocol $status $bytes_sent $bytes_received $session_time';
	
	access_log /var/log/nginx/access.log log_stream ;
	
	error_log  /var/log/nginx/error.log;
	
	upstream upstream_balancer {
		server 0.0.0.1:1234; # placeholder
		
		balancer_by_lua_block {
			tcp_udp_balancer.balance()
... skipping 25 lines ...
[ingress-nginx] Shutdown ingress controller
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:146
  should shutdown after waiting 60 seconds for pending connections to be closed [It]
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/gracefulshutdown/shutdown.go:71

  Expected
      <[]error | len:1, cap:1>: [
          {
              Op: "Get",
              URL: "http://10.96.224.148/sleep/70",
              Err: {
                  Op: "dial",
                  Net: "tcp",
... skipping 175 lines ...
STEP: Creating a kubernetes client
STEP: Building a namespace api object
Feb  5 01:43:29.396: INFO: Created namespace: e2e-tests-rewrite-1580867009388075185-drxkv
STEP: Starting new ingress controller
[BeforeEach] [ingress-nginx] Annotations - Rewrite
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/rewrite.go:35
[It] should fail to use longest match for documented warning
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/rewrite.go:164
STEP: creating a regular ingress definition
STEP: creating an ingress definition with the use-regex annotation
STEP: check that '/foo/bar/bar' does not match the longest exact path
[AfterEach] [ingress-nginx] Annotations - Rewrite
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
... skipping 2 lines ...
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/rewrite.go:39


• [SLOW TEST:54.355 seconds]
[ingress-nginx] Annotations - Rewrite
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:146
  should fail to use longest match for documented warning
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/rewrite.go:164
------------------------------
[BeforeEach] [ingress-nginx] Annotations - Auth
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:76
STEP: Creating a kubernetes client
STEP: Building a namespace api object
... skipping 29 lines ...
STEP: Adding an ingress rule for /foo
STEP: Adding an ingress rule for /bar
STEP: Adding an ingress rule for /foo
STEP: Adding an ingress rule for /bar
[It] should deny login for different location on same server
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/auth.go:471
STEP: receiving an internal server error without cache on location /bar
[AfterEach] [ingress-nginx] Annotations - Auth
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
STEP: Waiting for test namespace to no longer exist
[AfterEach] [ingress-nginx] Annotations - Auth
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/auth.go:43

... skipping 126 lines ...
		
		-- init modules
		local ok, res
		
		ok, res = pcall(require, "lua_ingress")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		lua_ingress = res
		lua_ingress.set_config({
			use_forwarded_headers = false,
			is_ssl_passthrough_enabled = false,
			http_redirect_code = 308,
... skipping 5 lines ...
			hsts_preload = false,
		})
		end
		
		ok, res = pcall(require, "configuration")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		configuration = res
		end
		
		ok, res = pcall(require, "balancer")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		balancer = res
		end
		
		ok, res = pcall(require, "monitor")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		monitor = res
		end
		
		ok, res = pcall(require, "certificate")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		certificate = res
		end
		
		ok, res = pcall(require, "plugins")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		plugins = res
		end
		-- load all plugins that'll be used here
	plugins.init({})
	}
... skipping 87 lines ...
		
		default 1;
	}
	
	access_log /var/log/nginx/access.log upstreaminfo  if=$loggable;
	
	error_log  /var/log/nginx/error.log notice;
	
	resolver 10.96.0.10 valid=30s;
	
	# See https://www.nginx.com/blog/websocket-nginx
	map $http_upgrade $connection_upgrade {
		default          upgrade;
... skipping 194 lines ...
			proxy_request_buffering                 on;
			proxy_http_version                      1.1;
			
			proxy_cookie_domain                     off;
			proxy_cookie_path                       off;
			
			# In case of errors try the next upstream server before returning an error
			proxy_next_upstream                     error timeout;
			proxy_next_upstream_timeout             0;
			proxy_next_upstream_tries               3;
			
			proxy_pass http://upstream_balancer;
			
			proxy_redirect                          off;
... skipping 4 lines ...
		location /healthz {
			
			access_log off;
			return 200;
		}
		
		# this is required to avoid error if nginx is being monitored
		# with an external software (like sysdig)
		location /nginx_status {
			
			allow 127.0.0.1;
			
			allow ::1;
... skipping 121 lines ...
			proxy_request_buffering                 on;
			proxy_http_version                      1.1;
			
			proxy_cookie_domain                     off;
			proxy_cookie_path                       off;
			
			# In case of errors try the next upstream server before returning an error
			proxy_next_upstream                     error timeout;
			proxy_next_upstream_timeout             0;
			proxy_next_upstream_tries               3;
			
			proxy_pass http://upstream_balancer;
			
			proxy_redirect                          off;
... skipping 76 lines ...
		
		-- init modules
		local ok, res
		
		ok, res = pcall(require, "configuration")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		configuration = res
		end
		
		ok, res = pcall(require, "tcp_udp_configuration")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		tcp_udp_configuration = res
		end
		
		ok, res = pcall(require, "tcp_udp_balancer")
		if not ok then
		error("require failed: " .. tostring(res))
		else
		tcp_udp_balancer = res
		end
	}
	
	init_worker_by_lua_block {
... skipping 3 lines ...
	lua_add_variable $proxy_upstream_name;
	
	log_format log_stream '[$remote_addr] [$time_local] $protocol $status $bytes_sent $bytes_received $session_time';
	
	access_log /var/log/nginx/access.log log_stream ;
	
	error_log  /var/log/nginx/error.log;
	
	upstream upstream_balancer {
		server 0.0.0.1:1234; # placeholder
		
		balancer_by_lua_block {
			tcp_udp_balancer.balance()
... skipping 25 lines ...
[ingress-nginx] Shutdown ingress controller
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:146
  should shutdown after waiting 60 seconds for pending connections to be closed [It]
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/gracefulshutdown/shutdown.go:71

  Expected
      <[]error | len:1, cap:1>: [
          {
              Op: "Get",
              URL: "http://10.96.194.109/sleep/70",
              Err: {
                  Op: "dial",
                  Net: "tcp",
... skipping 654 lines ...
STEP: Creating a kubernetes client
STEP: Building a namespace api object
Feb  5 01:48:19.970: INFO: Created namespace: e2e-tests-authtls-1580867299955182407-h26mn
STEP: Starting new ingress controller
[BeforeEach] [ingress-nginx] Annotations - AuthTLS
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/authtls.go:34
[It] should set valid auth-tls-secret, pass certificate to upstream, and error page
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/authtls.go:114
[AfterEach] [ingress-nginx] Annotations - AuthTLS
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
STEP: Waiting for test namespace to no longer exist
[AfterEach] [ingress-nginx] Annotations - AuthTLS
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/annotations/authtls.go:38
... skipping 111 lines ...
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:146
  given an ingress with TLS correctly configured [BeforeEach]
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/lua/dynamic_certificates.go:109
    removes HTTPS configuration when we delete TLS spec
    /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/lua/dynamic_certificates.go:224

    Unexpected error:
        <*errors.errorString | 0xc00019c030>: {
            s: "unexpected error waiting for ingress controller deployment: exit status 1.\nLogs:\ndeploying NGINX Ingress controller in namespace e2e-tests-dynamic-certificate-1580866765007282909-t2d2t\nserviceaccount/nginx-ingress-serviceaccount created\nrole.rbac.authorization.k8s.io/nginx-ingress-role created\nclusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole-e2e-tests-dynamic-certificate-1580866765007282909-t2d2t created\nrolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created\nclusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding-e2e-tests-dynamic-certificate-1580866765007282909-t2d2t created\nconfigmap/nginx-configuration created\nconfigmap/tcp-services created\nconfigmap/udp-services created\nservice/ingress-nginx created\ndeployment.apps/nginx-ingress-controller created\nlimitrange/ingress-nginx created\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nW0205 01:42:27.350212    5828 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server (\"unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)\") has prevented the request from succeeding\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nW0205 01:45:28.501202    5828 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server (\"unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)\") has prevented the request from succeeding\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nW0205 01:48:29.527178    5828 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server (\"unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)\") has prevented the request from succeeding\nWaiting for deployment \"nginx-ingress-controller\" rollout to finish: 0 of 1 updated replicas are available...\nerror: deployment \"nginx-ingress-controller\" exceeded its progress deadline\nObtaining ingress controller pod logs...\nI0205 01:46:12.816389       7 main.go:152] Received SIGTERM, shutting down\nI0205 01:46:12.816456       7 nginx.go:391] Shutting down controller queues\nI0205 01:46:12.816480       7 status.go:117] updating status of Ingress rules (remove)\nI0205 01:46:12.839946       7 status.go:136] removing address from ingress status ([10.96.14.76])\nI0205 01:46:12.840065       7 nginx.go:407] Stopping NGINX process\nW0205 01:46:13.516659       7 controller.go:180] Dynamic reconfiguration failed: Post http://127.0.0.1:10246/configuration/backends: dial tcp 127.0.0.1:10246: connect: connection refused\nE0205 01:46:13.516708       7 controller.go:184] Unexpected failure reconfiguring NGINX:\nPost http://127.0.0.1:10246/configuration/backends: dial tcp 127.0.0.1:10246: connect: connection refused\nW0205 01:46:13.516722       7 queue.go:130] requeuing initial-sync, err Post http://127.0.0.1:10246/configuration/backends: dial tcp 127.0.0.1:10246: connect: connection refused\n2020/02/05 01:46:12 [notice] 46#46: signal process started\n",
        }
        unexpected error waiting for ingress controller deployment: exit status 1.
        Logs:
        deploying NGINX Ingress controller in namespace e2e-tests-dynamic-certificate-1580866765007282909-t2d2t
        serviceaccount/nginx-ingress-serviceaccount created
        role.rbac.authorization.k8s.io/nginx-ingress-role created
        clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole-e2e-tests-dynamic-certificate-1580866765007282909-t2d2t created
        rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
... skipping 2 lines ...
        configmap/tcp-services created
        configmap/udp-services created
        service/ingress-nginx created
        deployment.apps/nginx-ingress-controller created
        limitrange/ingress-nginx created
        Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
        W0205 01:42:27.350212    5828 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server ("unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)") has prevented the request from succeeding
        Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
        W0205 01:45:28.501202    5828 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server ("unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)") has prevented the request from succeeding
        Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
        W0205 01:48:29.527178    5828 reflector.go:302] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server ("unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)") has prevented the request from succeeding
        Waiting for deployment "nginx-ingress-controller" rollout to finish: 0 of 1 updated replicas are available...
        error: deployment "nginx-ingress-controller" exceeded its progress deadline
        Obtaining ingress controller pod logs...
        I0205 01:46:12.816389       7 main.go:152] Received SIGTERM, shutting down
        I0205 01:46:12.816456       7 nginx.go:391] Shutting down controller queues
        I0205 01:46:12.816480       7 status.go:117] updating status of Ingress rules (remove)
        I0205 01:46:12.839946       7 status.go:136] removing address from ingress status ([10.96.14.76])
        I0205 01:46:12.840065       7 nginx.go:407] Stopping NGINX process
        W0205 01:46:13.516659       7 controller.go:180] Dynamic reconfiguration failed: Post http://127.0.0.1:10246/configuration/backends: dial tcp 127.0.0.1:10246: connect: connection refused
        E0205 01:46:13.516708       7 controller.go:184] Unexpected failure reconfiguring NGINX:
        Post http://127.0.0.1:10246/configuration/backends: dial tcp 127.0.0.1:10246: connect: connection refused
        W0205 01:46:13.516722       7 queue.go:130] requeuing initial-sync, err Post http://127.0.0.1:10246/configuration/backends: dial tcp 127.0.0.1:10246: connect: connection refused
        2020/02/05 01:46:12 [notice] 46#46: signal process started
        
    occurred
... skipping 213 lines ...
STEP: Starting new ingress controller
[BeforeEach] [ingress-nginx] Configmap change
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/configmap_change.go:32
[It] should reload after an update in the configuration
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/configmap_change.go:39
STEP: adding a whitelist-source-range
STEP: changing error-log-level
[AfterEach] [ingress-nginx] Configmap change
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
STEP: Waiting for test namespace to no longer exist
[AfterEach] [ingress-nginx] Configmap change
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/configmap_change.go:36

... skipping 102 lines ...
STEP: Creating a kubernetes client
STEP: Building a namespace api object
Feb  5 01:49:41.322: INFO: Created namespace: e2e-tests-default-ssl-certificate-1580867381317240696-5fdrp
STEP: Starting new ingress controller
[BeforeEach] [ingress-nginx] default-ssl-certificate
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/default_ssl_certificate.go:39
Feb  5 01:50:12.322: INFO: Unexpected TLS error: x509: certificate is valid for ingress.local, not *
Feb  5 01:50:15.322: INFO: Unexpected TLS error: x509: certificate is valid for ingress.local, not *
[It] uses default ssl certificate for host based ingress when configured certificate does not match host
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/settings/default_ssl_certificate.go:80
STEP: making sure new ingress is deployed
STEP: making sure the configured default ssl certificate is being used
[AfterEach] [ingress-nginx] default-ssl-certificate
  /home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:77
... skipping 590 lines ...
Feb  5 01:51:48.146: INFO: Running AfterSuite actions on all nodes



Summarizing 4 Failures:

[Fail] [ingress-nginx] X-Forwarded headers [BeforeEach] should trust X-Forwarded headers when setting is true 
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:106

[Fail] [ingress-nginx] Shutdown ingress controller [It] should shutdown after waiting 60 seconds for pending connections to be closed 
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/gracefulshutdown/shutdown.go:119

[Fail] [ingress-nginx] Shutdown ingress controller [It] should shutdown after waiting 60 seconds for pending connections to be closed 
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/gracefulshutdown/shutdown.go:119

[Fail] [ingress-nginx] Dynamic Certificate [BeforeEach] given an ingress with TLS correctly configured removes HTTPS configuration when we delete TLS spec 
/home/prow/go/src/k8s.io/ingress-nginx/test/e2e/framework/framework.go:106

Ran 211 of 213 Specs in 1490.272 seconds
FAIL! -- 210 Passed | 1 Failed | 2 Flaked | 0 Pending | 2 Skipped


Ginkgo ran 1 suite in 24m50.527617036s
Test Suite Failed
pod "e2e" deleted
pod default/e2e terminated (Error)
make[1]: *** [Makefile:195: e2e-test] Error 1
Exported logs for cluster "ingress-nginx-dev" to:
/logs/artifacts/logs
Deleting cluster "ingress-nginx-dev" ...
make: *** [Makefile:257: kind-e2e-test] Error 2
+ EXIT_VALUE=2
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
[Barnacle] 2020/02/05 01:54:02 Cleaning up Docker data root...
[Barnacle] 2020/02/05 01:54:02 Removing all containers.
... skipping 18 lines ...