[Bug 1561927] Re: Juju 1.25.3 - infinite loop on cluster-relation-changed - cinder/apache services constantly restarted

[OpenStack Infra]

Reviewed:  https://review.openstack.org/402954
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=1e1000a0892046b158e104025b04d3cf53a2a1b8
Submitter: Jenkins
Branch:    master

commit 1e1000a0892046b158e104025b04d3cf53a2a1b8
Author: Edward Hope-Morley <edward.hope-morley at canonical.com>
Date:   Fri Nov 25 16:20:05 2016 +0000

    Fix cluster relation unnecessary service restarts
    
    The logic introduced in commit 619ce065 to formalise database
    initialisation did not support the leader switching and re-runs
    of the shared-db relation. This resulted in extraneous service
    restarts. We avoid this by adding some extra logic around this
    code.
    
    Change-Id: If988331e552da930eff868abded323014fd50f04
    Closes-Bug: 1561927


** Changed in: cinder (Juju Charms Collection)
       Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to cinder in Juju Charms Collection.
Matching subscriptions: charm-bugs
https://bugs.launchpad.net/bugs/1561927

Title:
  Juju 1.25.3 - infinite loop on cluster-relation-changed -
  cinder/apache services constantly restarted

Status in cinder package in Juju Charms Collection:
  Fix Committed

Bug description:
  LSB: Ubuntu 14.04.4 LTS
  openstack: cloud:trusty-liberty
  cinder packages: 2:7.0.1-0ubuntu1~cloud0
  cinder charm: lp:charms/trusty/cinder;revno=106
  Juju: 1.25.3.1
  num_units: 3
  related to hacluster charm (which remains idle)

  Symptoms:
  All three units constantly run cluster-relation-changed, causing restart of all cinder upstart jobs as well as apache2.
  If a unit is stopped, the other two stop looping. By restarting stopped unit and juju resolving it, loop starts on all three.

  Temp solution to end loop (only applied on cinder/1):
  """
  @hooks.hook('cluster-relation-changed',
              'cluster-relation-departed')
  @restart_on_change(restart_map(), stopstart=True)
  def cluster_changed():
      #check_db_initialised()
      #CONFIGS.write_all()
      pass
  """

  Such temporary solution was done at 13:08 (see attached 20160325-unit-
  cinder-1.log). Once all three units settled, I rolledback
  cluster_changed() config to original code (uncommenting
  check_db_initialised and CONFIGS.write_all), at 13:09.

  Please let me know if you need further details.

To manage notifications about this bug go to:
https://bugs.launchpad.net/charms/+source/cinder/+bug/1561927/+subscriptions