[Bug 1620897] Re: Fix race on getting close notifier channel

[Andy Whitcroft]

Hello Xiang, or anyone else affected,

Accepted etcd into xenial-proposed. The package will build now and be
available at https://launchpad.net/ubuntu/+source/etcd/2.2.5+dfsg-
1ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: etcd (Ubuntu Xenial)
       Status: New => Fix Committed

** Tags added: verification-needed

** Changed in: etcd (Ubuntu)
       Status: Confirmed => Fix Committed

** Changed in: etcd (Ubuntu Xenial)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1620897

Title:
  Fix race on getting close notifier channel

Status in etcd package in Ubuntu:
  Fix Committed
Status in etcd source package in Xenial:
  Fix Committed

Bug description:
  [OS]
  Ubuntu Xenial
  etcd 2.2.5

  [Error Log]
  Sep 2 16:08:16 ubuntu etcd[18180]: panic: net/http: CloseNotify called after ServeHTTP finished
  Sep 2 16:08:16 ubuntu etcd[18180]: goroutine 421 [running]:
  Sep 2 16:08:16 ubuntu etcd[18180]: net/http.(*response).CloseNotify(0xc8202f64e0, 0x0)
  Sep 2 16:08:16 ubuntu etcd[18180]: #011/usr/lib/go/src/net/http/server.go:1535 +0x9d
  Sep 2 16:08:16 ubuntu etcd[18180]: github.com/coreos/etcd/proxy.(*reverseProxy).ServeHTTP.func1(0x7fb0024d1a80, 0xc8202f64e0, 0xc8203e4f78, 0xc8202e8a80, 0xc8203e4fa0, 0xc820464f50)
  Sep 2 16:08:16 ubuntu etcd[18180]: #011/build/etcd-tG_CNV/etcd-2.2.5+dfsg/obj-x86_64-linux-gnu/src/github.com/coreos/etcd/proxy/reverse.go:107 +0x39
  Sep 2 16:08:16 ubuntu etcd[18180]: created by github.com/coreos/etcd/proxy.(*reverseProxy).ServeHTTP
  Sep 2 16:08:16 ubuntu etcd[18180]: #011/build/etcd-tG_CNV/etcd-2.2.5+dfsg/obj-x86_64-linux-gnu/src/github.com/coreos/etcd/proxy/reverse.go:113 +0x691
  Sep 2 16:08:16 ubuntu systemd[1]: etcd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
  Sep 2 16:08:16 ubuntu systemd[1]: etcd.service: Unit entered failed state.
  Sep 2 16:08:16 ubuntu systemd[1]: etcd.service: Failed with result 'exit-code'.

  etcd proxy process dies and is not restarted by systemd, Components
  that depend on etcd report connection errors like this:

  - Request to server http://127.0.0.1:4001 failed:
  MaxRetryError("HTTPConnectionPool(host='127.0.0.1', port=4001): Max
  retries exceeded with url:
  /v2/keys/calico/dhcp/v1/subnet?waitIndex=1625&recursive=true&wait=true
  (Caused by NewConnectionError('<urllib3.connection.HTTPConnection
  object at 0x7f6f0c5cae10>: Failed to establish a new connection:
  [Errno 111] ECONNREFUSED',))",)

  [Root Cause]
  etcd proxy panic occur in etcd 2.2.5, which is fixed in etcd 2.3.4 (https://github.com/coreos/etcd/pull/5269/files#r62072134), currently only have 2.2.5 in xenial, we need it backported.

  -------------------------------------------------------

  [Impact]
  This patch fix race on getting close notifier channel when a panic reported as 'net/http: CloseNotify called after ServeHTTP finished'.

  [Test Case]
  No special configuration, running etcd in proxy mode.

  [Regression Potential]
  etcd no longer crashes every so often while in proxy mode...

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/etcd/+bug/1620897/+subscriptions