[Bug 1879798] Re: replacing designate units causes issues previously created zones

[Jorge Niedbalski]

### Further observations ####

I am able to partially reproduce the problem.

Bundle used: http://paste.ubuntu.com/p/myxQJnJvyn/


$ openstack zone create --email dnsmaster at example.com example.com.
$ rndc showzone example.com.

zone "example.com" { type slave; file "slave.example.com.f3e3fdaa-
857e-4786-afef-2b4cb2d03357"; masters { 10.5.0.10 port 5354; 10.5.0.41
port 5354; 10.5.0.31 port 5354; }; };


$ juju remove-unit designate/0 designate/1 designate/2 --force
removing unit designate/0
removing unit designate/1
removing unit designate/2


root at juju-54f98f-1879798-4:/home/ubuntu# ack master /var/log/syslog 
May 27 19:21:20 juju-54f98f-1879798-4 named[1505]: received control channel command 'addzone example.com  { type slave; masters { 10.5.0.10 port 5354; 10.5.0.41 port 5354; 10.5.0.31 port 5354;}; file "slave.example.com.f3e3fdaa-857e-4786-afef-2b4cb2d03357"; };'
May 27 19:55:32 juju-54f98f-1879798-4 named[6653]: zone example.com/IN: refresh: timeout retrying without EDNS master 10.5.0.10#5354 (source 0.0.0.0#0)
May 27 19:55:47 juju-54f98f-1879798-4 named[6653]: zone example.com/IN: refresh: retry limit for master 10.5.0.10#5354 exceeded (source 0.0.0.0#0)
May 27 19:56:05 juju-54f98f-1879798-4 named[6653]: zone example.com/IN: refresh: retry limit for master 10.5.0.41#5354 exceeded (source 0.0.0.0#0)
May 27 19:56:23 juju-54f98f-1879798-4 named[6653]: zone example.com/IN: refresh: retry limit for master 10.5.0.31#5354 exceeded (source 0.0.0.0#0)


$ juju add-unit -n 3 designate


root at juju-54f98f-1879798-4:/home/ubuntu# ack addzone /var/log/syslog 
May 27 19:21:20 juju-54f98f-1879798-4 named[1505]: received control channel command 'addzone example.com  { type slave; masters { 10.5.0.10 port 5354; 10.5.0.41 port 5354; 10.5.0.31 port 5354;}; file "slave.example.com.f3e3fdaa-857e-4786-afef-2b4cb2d03357"; };'
May 27 19:21:20 juju-54f98f-1879798-4 named[1505]: added zone example.com in view _default via addzone

---

(Continues).

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to designate in Ubuntu.
https://bugs.launchpad.net/bugs/1879798

Title:
  replacing designate units causes issues previously created zones

Status in OpenStack Designate Charm:
  Confirmed
Status in OpenStack Designate-Bind Charm:
  Confirmed
Status in designate package in Ubuntu:
  New

Bug description:
  We have a designate/designate-bind setup. We migrated designate units
  to different machines, replacing 3 designate units with 3 new units.
  However, this caused issues with existing zones, including creating
  new recordsets for these zones. The zone would result in having an
  ERROR status and a CREATE action.

  Looking at the designate bind units, we see that designate is
  attempting to run:

  'addzone $zone { type slave; masters {$new_designate_ips port 5354;};
  file "slave.$zone.$hash"; };'

  This addzone fails due to the zone already existing. However, we found
  that the zone configuration (using 'rndc showzone $zone' from
  designate-bind unit) still had the old designate ips for its masters.
  There are also logs in /var/log/syslog like the following:

  May 20 06:27:10 juju-c27f05-15-lxd-1 named[72648]: transfer of '$zone'
  from $old_designate_ip#5354: failed to connect: host unreachable

  We were able to resolve this issue by modifying the zone config on all
  designate-bind units:

  juju run -a designate-bind -- rndc modzone $zone '{ type slave; file
  "slave.$zone.$hash"; masters { $new_designate_ip_1 port 5354;
  $new_designate_ip_2 port 5354; $new_designate_ip_3 port 5354; }; };'

  After modifying the zone, the recordset creations completed and
  resolved almost immediately.

  Would this be something the charm could do in an automated way when
  masters are removed/replaced, or is there a better way of fixing the
  zone configurations? For these designate migrations, we will have to
  enumerate over every zone to fix their configurations.

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-designate/+bug/1879798/+subscriptions