Persistence in charms

[Gary Poster]

On 02/24/12 09:36, Gustavo Niemeyer wrote:
> Hey Gary,

Hey.  Thanks for the reply, Gustavo.

>> Moreover, wanting a public url for these kinds of persistent files--logs,
>> database files and the like--seems like it would be the exception rather
>> than the rule.  Having them shared privately seems more commonly desirable.
>
> It sounds like you're asking for a file server.. this fits quite well
> as just another charm that your charm can require. It can be as simple
> as a webdav server that provides an authenticated https base URL
> through the relation data, and as complex as a deployment of ceph's
> object store.
>
> Wouldn't that solve your problem?

Yes, but I feel like I might not fully understand your point.

A file server would solve it, as would a shared directory on the host 
for an LXC instance, as would an S3 configured and ready to go on EC2, 
as would...any number of things, as you point out below.

>> Unless I misunderstand, they cannot abstract away the environment--lxc or
>> ec2 or maas.  In our discussions, we think charms ought to be able to say
>> "stash this file" and "give me back this file" without having to worry about
>> whether the stashing happened in an S3 bucket, a directory somewhere in the
>> LXC host's filesystem, or whatever the maas equivalent is.
>
> Right, that sounds like a perfect fit for a well defined "stash"
> relation that charms can depend on.

I agree, that sounds good a subordinate charm relation.  It would be a 
useful abstraction, and it would support your very good point below: 
people may want a variety of stash strategies.

However, I think it is a complement, not a replacement, for lower-level 
support in Juju such as what I described.

Consider the case of an LXC instance.  The simplest "stash" charm that 
could possibly work in that case would be a directory on the host 
system.  This charm would ideally be easy to write, and would be easy to 
use (for instance, as a user, you would not have to set up a separate 
file server).

Without support from Juju itself, this presents problems.  The charm 
would have to have knowledge of how to access the host system, which 
would almost certainly expose internals that will be shut down in a 
security audit of Juju later.  Additionally, the charm would have to be 
storage-dependent--a new and unattractive idea in Juju, AFAIK.

If instead the charm had access to a persistent storage location (or 
persistent storage commands), handled in the "natural" way for the given 
environment, and as configured in environments.yaml, this would mean 
that a simple "stash" charm could be written in an easy way, with very 
easy usage.  If someone has more specific persistence requirements, they 
can use a different implementation of the stash charm that does not rely 
on these Juju-provided but relies on external machinery.

A very similar argument could be made with EC2 and S3.

>> The approach to this that I like best would be for Juju to simply provide a
>> mounted directory in a known/documented location that promises to be a
>> persistent store for the machine, if such has been configured. The charm
>> simply looks for that directory, and uses it as it wishes. That could be
>> easy to use, powerful, and fairly easy to implement (I suspect).  It would
>> not even require writing any hooks.
>
> People will have different requirements for the storage depending on
> what they're doing. I don't think we should try solve that problem
> internally for everybody in a constrained way.

(Yes, that's compelling, and my reply above reflects this.)

>> Alternatively, we could have something like the two commands you mentioned,
>> but put-file would not return anything, and get-file would simply take the
>> same name.  For instance, if you wanted to stash
>
> get-file was discussed at UDS in a different context. It aims to solve
> a different problem than the one you're looking at.

Good, that jibes with my understanding of Clint's email then.  That 
said, we could still have commands of a different name that did what I 
described.  But that's not a first choice solution for either of us, so 
probably not worth discussing further.

Gary