The PostgreSQL charm, AWS and robustness

Wed May 28 15:00:21 UTC 2014

>> For the future, I believe there is work underway to replace most of
this with a common subordinate charm. I have even approved a merge
proposal for this, but merging it got blocked by getting the
subordinate charm landed (?). It seems much, much nicer. So far I've
left this code as I found it, since it is in production and written by
the people actually using it :)

As Stuart mentioned, there are changes in-review for the postgresql charm
specifically that were blocked on awaiting the release of two charms in the
charm store. The landscape team wants to use these to persist our DBs and
potentially use shared NFS to host our hosted repositories and logs.

The charms are now released thanks to David Britton's charm development and
driving these through the review process.

  - storage subordinate charm:
https://jujucharms.com/sidebar/search/precise/storage-0/?text=storage
       Generic subordinate charm which interacts either with related NFS or
block-storage-broker charms to request and mount persistent storage on its
principal unit

  - block-storage-broker charm:
https://jujucharms.com/sidebar/search/precise/block-storage-broker-2/?text=block-storage-broker
     Principal charm which brokers all interaction with either Nova or EC2
storage services to create, name and attach or detach persistent volumes
for juju units.

These charms are intended for general use, but the first example, other
than the landscape charm, is the postgresql MP below which will also
describes an example postgresql bundle and charm changes to support
persistent storage.
https://code.launchpad.net/~chad.smith/charms/precise/postgresql/postgresql-using-storage-subordinate/+merge/206078

The interaction of principal, storage subordinate and block-storage-broker
with a principal like postgresql:
 - block-storage-broker is configured talk to openstack or ec2
 - storage subordinate is related to block-storage-broker and configured to
use provider: block-storage-broker for any volume requests
 - storage subordinate is related to postgresql (or any other principal
that supports the block-storage interface)
 - principal (postgresql) specifies a volume mountpoint to storage
subordinate via relation data
  - storage interacts with block-storage-broker to reattach or create the
labelled volume if none exists and attach it to the postgresql unit
 - postgresql migrates data to the persistent device if necessary

Upon service unit failure:
-   The storage subordinate takes care of tagging and labeling the volume
created to tie that volume to a particular service unit by unit name
"postgresql/0". If the unit fails, the volume will still exist detached.
Upon a successful restart of the unit the relation between storage and
block-storage-broker will remount that persistent volume and use it.  Also,
if known storage volume-ids need to be mounted to specific units of service
the storage subordinate supports a configuring a "volume_map" YAML allowing
you to map {postgres/1: vol-0000016 ...}.

David did a lot of the design and implementation of the storage charms. He
also was the driver through the review process to get these charms into the
store with charmers. So, we should be able to answer questions about the
direction of either of these charms if they don't quite fit the use case.

Hope that helps,
Chad

>
>
> > How do I restore the database from a backup?
>
> To manually restore a backup, you 'juju ssh' into the unit, sudo to
> the postgres user, and run pg_restore(1) to load the dump. This is
> standard PostgreSQL, rather than anything specific to the charm. Given
> the flexibility of restoring PostgreSQL dumps, the charm could provide
> some simple actions (restore all, restore full db foo into bar), but
> the more common tasks of restoring small pieces are going to remain
> manual.
>
> Automatic restore is quite scary, since restoration is a data loss
> operation (from the perspective of any existing data being
> overwritten). It is also something you should never have to do.
> Replication and/or bringing up a new unit with the existing mount
> covers most of the traditional disaster recovery scenarios. Backups
> should really only be needed to recover from partial data loss
> (deleting something you shouldn't), total data loss (corrupted file
> system), or revisiting the past.
>
>
> > If the machine dies, which I will manually do in testing, how do I go
> > about bringing up a new machine, attaching the storage, deploying the
> > charm and have it use the existing database that is there?  I'm guessing
> > that there are orderings I need.
>
> No orderings needed. Bring up a new unit and then set volume-map, or
> set volume-map and then bring up the new unit. Whatever ;)
>
> If the settings are screwed up, the hook aborts and you will need to
> resolve the situation. juju could help here by allowing hooks more
> flexibility than reporting just 'success/fail'. My life would be
> easier with 'retry' (hook is waiting on hook on other units to run, or
> a required operation is blocked) and 'invalid config' (retry this hook
> after config-changed has successfully been run).
>
>
> > Do I have to deploy a new database with a new persistent mount point and
> > restore from a backup?  If I have to do that, what is the benefit of
> > having the database itself in a persistent mount?
> >
> > Is there someone knowledgeable on the postgres charm and AWS that can
> > help me with these points?
> >
> > BTW, I'm very happy to blog about this whole process once I have it all
> > figured out :-)
>
> Or give me a MP with an updated README :)
>
> --
> Stuart Bishop <stuart.bishop at canonical.com>
>
> --
> Juju mailing list
> Juju at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20140528/e50dee0a/attachment.html>