[Bug 1620897] Re: Fix race on getting close notifier channel

[Mathew Hodson]

** Changed in: etcd (Ubuntu)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1620897

Title:
  Fix race on getting close notifier channel

Status in etcd package in Ubuntu:
  Confirmed

Bug description:
  [OS]
  Ubuntu Xenial
  etcd 2.2.5

  
  [Error Log]
  Sep 2 16:08:16 ubuntu etcd[18180]: panic: net/http: CloseNotify called after ServeHTTP finished 
  Sep 2 16:08:16 ubuntu etcd[18180]: goroutine 421 [running]: 
  Sep 2 16:08:16 ubuntu etcd[18180]: net/http.(*response).CloseNotify(0xc8202f64e0, 0x0) 
  Sep 2 16:08:16 ubuntu etcd[18180]: #011/usr/lib/go/src/net/http/server.go:1535 +0x9d 
  Sep 2 16:08:16 ubuntu etcd[18180]: github.com/coreos/etcd/proxy.(*reverseProxy).ServeHTTP.func1(0x7fb0024d1a80, 0xc8202f64e0, 0xc8203e4f78, 0xc8202e8a80, 0xc8203e4fa0, 0xc820464f50) 
  Sep 2 16:08:16 ubuntu etcd[18180]: #011/build/etcd-tG_CNV/etcd-2.2.5+dfsg/obj-x86_64-linux-gnu/src/github.com/coreos/etcd/proxy/reverse.go:107 +0x39 
  Sep 2 16:08:16 ubuntu etcd[18180]: created by github.com/coreos/etcd/proxy.(*reverseProxy).ServeHTTP 
  Sep 2 16:08:16 ubuntu etcd[18180]: #011/build/etcd-tG_CNV/etcd-2.2.5+dfsg/obj-x86_64-linux-gnu/src/github.com/coreos/etcd/proxy/reverse.go:113 +0x691 
  Sep 2 16:08:16 ubuntu systemd[1]: etcd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT 
  Sep 2 16:08:16 ubuntu systemd[1]: etcd.service: Unit entered failed state. 
  Sep 2 16:08:16 ubuntu systemd[1]: etcd.service: Failed with result 'exit-code'.

  etcd proxy process dies and is not restarted by systemd, Components
  that depend on etcd report connection errors like this:

  - Request to server http://127.0.0.1:4001 failed:
  MaxRetryError("HTTPConnectionPool(host='127.0.0.1', port=4001): Max
  retries exceeded with url:
  /v2/keys/calico/dhcp/v1/subnet?waitIndex=1625&recursive=true&wait=true
  (Caused by NewConnectionError('<urllib3.connection.HTTPConnection
  object at 0x7f6f0c5cae10>: Failed to establish a new connection:
  [Errno 111] ECONNREFUSED',))",)

  
  [Root Cause]
  etcd proxy panic occur in etcd 2.2.5, which is fixed in etcd 2.3.4 (https://github.com/coreos/etcd/pull/5269/files#r62072134), currently only have 2.2.5 in xenial, we need it backported.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/etcd/+bug/1620897/+subscriptions