Using subdocument _id fields for multi-environment support

[Menno Smits]

(doh...hit send accidentally)

TL;DR: It has been surprisingly difficult to use subdocument ids within
Juju and we are abandoning this approach. As per what has already been done
for the units and services collections, we will continue with the approach
of using "<uuid>:<id>" style string ids while also adding separate env-UUID
and collection specific identifier fields.


Jesse and I have been making the changes to use subdocument ids for the
units and services collections this week at the sprint and we've come up
against some unexpected issues.

We found that one part of the mgo/txn package wasn't happy using struct ids
and have been working with Gustavo to fix that. This isn't a show-stopper
but has slowed us down.

We also found unexpected friction with the implementation of the watchers
and entity life. These areas deeply assume that our document ids are
strings and fixing them requires wide-ranging and often ugly changes which
will take significant time to get right. It's been brick wall after brick
wall. We discussed with Tim, Will, John and Ian yesterday and given that
it's important that multi-environment support lands soon and given that the
watchers are going to completely change in the not too distant future[1],
we have abandoned the approach of using subdocument ids for
multi-environment support. The benefits of using subdocument ids are
outweighed by the amount of change required to get there.

- Menno

[1] opening up the possibility of surrogate keys as document ids, where we
need application domain fields to exist fields outside of the _id.


On 10 October 2014 10:09, Menno Smits <menno.smits at canonical.com> wrote:

> TL;DR: It has been surprisingly difficult As per what has already been
> done for the units and services collections, we will continue with the
> approach of using "<uuid>:<id>" style string ids while also adding separate
> env-UUID and collection specific identifier fields.
>
> Jesse and I have been making the changes to use subdocument ids for the
> units and services collections this week at the sprint and we've come up
> against some unexpected issues.
>
> We found that one part of the mgo/txn package wasn't happy using struct
> ids and have been working with Gustavo to fix that. This isn't a
> show-stopper but has slowed us down.
>
> We also found unexpected friction with the implementation of the watchers
> and entity life. These areas deeply assume that our document ids are
> strings and fixing them requires wide-ranging and often ugly changes which
> will take significant time to get right. It's been brick wall after brick
> wall. We discussed with Tim, Will, John and Ian yesterday and given that
> it's important that multi-environment support lands soon and given that the
> watchers are going to completely change in the not too distant future[1],
> we have abandoned the approach of using subdocument idfs for
> multi-environment support. The benefits of using subdocuments ids are
> outweighed by the chan
>
>
>
> - Menno
>
> [1] opening up the possibility of surrogate keys as document ids, where we
> need application domain fields to exist fields outside of the _id.
>
>
> On 1 October 2014 22:11, Menno Smits <menno.smits at canonical.com> wrote:
>
>>
>>
>> On 2 October 2014 01:31, Kapil Thangavelu <kapil.thangavelu at canonical.com
>> > wrote:
>>
>>> it feels a little strange to use a mutable object for an immutable
>>> field. that said it does seem functional. although the immutability speaks
>>> to the first disadvantage noted for the separate fields namely becoming out
>>> of sync, which afaics isn't something that's possible with the current
>>> model, ie. a change of name needs to generate a new doc. Names (previous
>>> _id) are unique in usage minus the extant bug that unit ids are reused.
>>> even without that the benefits to avoiding the duplicate doc data and
>>> manual parse on every _id seem like clear wins for subdoc _ids.
>>>
>>
>> Just to be really sure, I added a test that exercises the case of one of
>> the _id fields changing. See TestAttemptedIdUpdate in the (just updated)
>> gist. MongoDB stops us from doing anything stupid (as expected).
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20141010/713e6096/attachment.html>