[Bug 1476428] [NEW] haproxy service handling failing to stop old instances

JuanJo Ciarlante 1476428 at bugs.launchpad.net
Mon Jul 20 23:43:58 UTC 2015 

Public bug reported:

On an openstack HA kilo deployment using charms trunks,
several services failing to properly restart haproxy, leaving
old instances running, showing cinder/0 as example:

$ juju ssh cinder/0 'pgrep -f haproxy | xargs ps -o pid,ppid,lstart,cmd -p; egrep St.*ing.haproxy /var/log/juju/unit-cinder-0.log'
    PID    PPID                  STARTED CMD
  13913       1 Sun Jul 19 01:02:11 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  14448       1 Sun Jul 19 01:04:59 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  14848       1 Sun Jul 19 01:05:03 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  21437       1 Sun Jul 19 01:07:00 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  27656       1 Sun Jul 19 01:08:33 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  32073       1 Sun Jul 19 01:09:27 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  39752       1 Sun Jul 19 01:10:59 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  39829       1 Sun Jul 19 01:11:00 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
  44200       1 Sun Jul 19 01:18:57 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
 299019  299018 Mon Jul 20 23:38:30 2015 bash -c pgrep -f haproxy | xargs ps -o pid,ppid,lstart,cmd -p; egrep St.*ing.haproxy /var/log/juju/unit-cinder-0.log
2015-07-19 01:02:14 INFO config-changed  * Stopping haproxy haproxy
2015-07-19 01:02:15 INFO config-changed  * Starting haproxy haproxy
2015-07-19 01:05:03 INFO cluster-relation-changed  * Stopping haproxy haproxy
2015-07-19 01:05:03 INFO cluster-relation-changed  * Starting haproxy haproxy
2015-07-19 01:05:07 INFO cluster-relation-changed  * Stopping haproxy haproxy
2015-07-19 01:05:07 INFO cluster-relation-changed  * Starting haproxy haproxy
2015-07-19 01:11:03 INFO identity-service-relation-changed  * Stopping haproxy haproxy
2015-07-19 01:11:03 INFO identity-service-relation-changed  * Starting haproxy haproxy
(copied also to http://paste.ubuntu.com/11911818/)

As shown above, new haproxy instances ~correlate with
Starting/Stopping lines at juju log, as expected.

FYI same issue find for (HA) services using haproxy, positively confirmed on:
nova-cloud-controller, keystone,  glance, cinder,
openstack-dashboard, swift-proxy, ceilometer

** Affects: ceilometer (Juju Charms Collection)
     Importance: Undecided
         Status: New

** Affects: cinder (Juju Charms Collection)
     Importance: Undecided
         Status: New

** Affects: glance (Juju Charms Collection)
     Importance: Undecided
         Status: New

** Affects: keystone (Juju Charms Collection)
     Importance: Undecided
         Status: New

** Affects: neutron-api (Juju Charms Collection)
     Importance: Undecided
         Status: New

** Affects: nova-cloud-controller (Juju Charms Collection)
     Importance: Undecided
         Status: New

** Affects: openstack-dashboard (Juju Charms Collection)
     Importance: Undecided
         Status: New

** Affects: swift-proxy (Juju Charms Collection)
     Importance: Undecided
         Status: New


** Tags: canonical-bootstack canonical-is

** Also affects: keystone (Juju Charms Collection)
   Importance: Undecided
       Status: New

** Also affects: cinder (Juju Charms Collection)
   Importance: Undecided
       Status: New

** Also affects: glance (Ubuntu)
   Importance: Undecided
       Status: New

** No longer affects: glance (Ubuntu)

** Also affects: glance (Juju Charms Collection)
   Importance: Undecided
       Status: New

** Also affects: openstack-dashboard (Juju Charms Collection)
   Importance: Undecided
       Status: New

** Also affects: neutron-api (Juju Charms Collection)
   Importance: Undecided
       Status: New

** Also affects: swift-proxy (Juju Charms Collection)
   Importance: Undecided
       Status: New

** Also affects: ceilometer (Juju Charms Collection)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to glance in Ubuntu.
https://bugs.launchpad.net/bugs/1476428

Title:
  haproxy service handling failing to stop old instances

Status in ceilometer package in Juju Charms Collection:
  New
Status in cinder package in Juju Charms Collection:
  New
Status in glance package in Juju Charms Collection:
  New
Status in keystone package in Juju Charms Collection:
  New
Status in neutron-api package in Juju Charms Collection:
  New
Status in nova-cloud-controller package in Juju Charms Collection:
  New
Status in openstack-dashboard package in Juju Charms Collection:
  New
Status in swift-proxy package in Juju Charms Collection:
  New

Bug description:
  On an openstack HA kilo deployment using charms trunks,
  several services failing to properly restart haproxy, leaving
  old instances running, showing cinder/0 as example:

  $ juju ssh cinder/0 'pgrep -f haproxy | xargs ps -o pid,ppid,lstart,cmd -p; egrep St.*ing.haproxy /var/log/juju/unit-cinder-0.log'
      PID    PPID                  STARTED CMD
    13913       1 Sun Jul 19 01:02:11 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    14448       1 Sun Jul 19 01:04:59 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    14848       1 Sun Jul 19 01:05:03 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    21437       1 Sun Jul 19 01:07:00 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    27656       1 Sun Jul 19 01:08:33 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    32073       1 Sun Jul 19 01:09:27 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    39752       1 Sun Jul 19 01:10:59 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    39829       1 Sun Jul 19 01:11:00 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
    44200       1 Sun Jul 19 01:18:57 2015 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -D -p /var/run/haproxy.pid
   299019  299018 Mon Jul 20 23:38:30 2015 bash -c pgrep -f haproxy | xargs ps -o pid,ppid,lstart,cmd -p; egrep St.*ing.haproxy /var/log/juju/unit-cinder-0.log
  2015-07-19 01:02:14 INFO config-changed  * Stopping haproxy haproxy
  2015-07-19 01:02:15 INFO config-changed  * Starting haproxy haproxy
  2015-07-19 01:05:03 INFO cluster-relation-changed  * Stopping haproxy haproxy
  2015-07-19 01:05:03 INFO cluster-relation-changed  * Starting haproxy haproxy
  2015-07-19 01:05:07 INFO cluster-relation-changed  * Stopping haproxy haproxy
  2015-07-19 01:05:07 INFO cluster-relation-changed  * Starting haproxy haproxy
  2015-07-19 01:11:03 INFO identity-service-relation-changed  * Stopping haproxy haproxy
  2015-07-19 01:11:03 INFO identity-service-relation-changed  * Starting haproxy haproxy
  (copied also to http://paste.ubuntu.com/11911818/)

  As shown above, new haproxy instances ~correlate with
  Starting/Stopping lines at juju log, as expected.

  FYI same issue find for (HA) services using haproxy, positively confirmed on:
  nova-cloud-controller, keystone,  glance, cinder,
  openstack-dashboard, swift-proxy, ceilometer

To manage notifications about this bug go to:
https://bugs.launchpad.net/charms/+source/ceilometer/+bug/1476428/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list