** Also affects: openstack-dashboard (Juju Charms Collection)
Importance: Undecided
Status: New
** Also affects: ceph-radosgw (Juju Charms Collection)
Importance: Undecided
Status: New
** Also affects: swift-proxy (Juju Charms Collection)
Importance: Undecided
Status: New
** Changed in: ceph-radosgw (Juju Charms Collection)
Status: New => Confirmed
** Changed in: ceph-radosgw (Juju Charms Collection)
Importance: Undecided => High
** Changed in: openstack-dashboard (Juju Charms Collection)
Status: New => Confirmed
** Changed in: openstack-dashboard (Juju Charms Collection)
Importance: Undecided => High
** Changed in: swift-proxy (Juju Charms Collection)
Status: New => Confirmed
** Changed in: swift-proxy (Juju Charms Collection)
Importance: Undecided => High
** Changed in: openstack-dashboard (Juju Charms Collection)
Milestone: None => 16.01
** Changed in: swift-proxy (Juju Charms Collection)
Milestone: None => 16.01
** Changed in: ceph-radosgw (Juju Charms Collection)
Milestone: None => 16.01
** Branch linked: lp:~thedac/charms/trusty/nova-cloud-
controller/lp1522130
** Branch linked: lp:~thedac/charms/trusty/neutron-api/lp1522130
** Branch linked: lp:~thedac/charms/trusty/keystone/lp1522130
** Branch linked: lp:~thedac/charms/trusty/glance/lp1522130
** Branch linked: lp:~thedac/charms/trusty/cinder/lp1522130
** Branch linked: lp:~thedac/charms/trusty/openstack-dashboard/lp1522130
** Branch linked: lp:~thedac/charms/trusty/ceph-radosgw/lp1522130
** Branch linked: lp:~thedac/charms/trusty/swift-proxy/lp1522130
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to cinder in Juju Charms Collection.
Matching subscriptions: charm-bugs
https://bugs.launchpad.net/bugs/1522130
Title:
Openstack API Charms have a naive haproxy connection timeout
Status in ceph-radosgw package in Juju Charms Collection:
Confirmed
Status in cinder package in Juju Charms Collection:
Confirmed
Status in glance package in Juju Charms Collection:
Confirmed
Status in keystone package in Juju Charms Collection:
Confirmed
Status in neutron-api package in Juju Charms Collection:
Confirmed
Status in nova-cloud-controller package in Juju Charms Collection:
Confirmed
Status in openstack-dashboard package in Juju Charms Collection:
Confirmed
Status in swift-proxy package in Juju Charms Collection:
Confirmed
Bug description:
On OpenStack API charms haproxy sets timeout connection to 1 second.
This is reasonable only in an ideal scenario. Connections to backends
on a busy cloud may take much longer. Even haproxy docs suggest 4 to 5
seconds [1].
defaults
log global
mode tcp
option tcplog
option dontlognull
retries 3
timeout queue 1000
timeout connect 1000
This causes mysterious errors like the following:
On neutron-api:
ConnectionRefused: Unable to establish connection to http://10.245.161.159:9696/v2.0/security-groups.json
On keystone:
ERROR: HTTPConnectionPool(host='10.5.51.133', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'httplib.BadStatusLine'>: '')
On nova-cloud-controller:
requests.exceptions.ConnectionError: HTTPConnectionPool(host='10.245.161.158', port=8774): Max retries exceeded with url: /v2/ba51369fa6444c6d9c2be7264675741f/servers/373ed532-9819-4f2b-b32a-4d27147d0338/os-interface (Caused by <class 'httplib.BadStatusLine'>: '')
When max_retries is forced > 1 it becomes only slightly less opaque:
urllib3.connectionpool - WARNING - Retrying (5 attempts remain) after
connection broken by 'BadStatusLine("''",)':
/v2/ba51369fa6444c6d9c2be7264675741f/servers/32eed1a0-6c9d-497f-b9fb-
aac0fa748858/os-interface
Set timeout connect (and timeout queue while we are at it) to sane
defaults and optionally make this configurable.
[1] https://cbonte.github.io/haproxy-
dconv/configuration-1.5.html#4-timeout%20connect
To manage notifications about this bug go to:
https://bugs.launchpad.net/charms/+source/ceph-radosgw/+bug/1522130/+subscriptions