RFC: mongo "_id" fields in the multi-environment juju server world

[roger peppe]

On 4 July 2014 15:17, Gustavo Niemeyer <gustavo at niemeyer.net> wrote:
> On Fri, Jul 4, 2014 at 10:32 AM, roger peppe <roger.peppe at canonical.com> wrote:
>> It won't be possible to shard the transaction log.
>
> Why not?

I had assumed that because every client needs to see every transaction
there would likely be no benefit to sharding the log, although
technically you could shard on transaction id. I'd be
delighted to be shown that my assumption is wrong though.
Perhaps the round-robin might really help.

>> The thing I'm trying to get across is: until we know one way or
>> another, I believe it would be better to choose the (much) simpler
>> option and use the (potential weeks of) dev time for other things.
>
> We know it's a bad idea. Besides everything else I mentioned, there
> are _huge_ MongoDB databases out there being that depend on sharding
> to scale.. we're talking hundreds of machines. It seems very naive to
> go with a model that loses the benefits of all the lessons the MongoDB
> development team learned with those use cases, and the work they have
> done to support them well.
>
> We have been there in Canonical. Ask folks about the CouchDB story.

Thanks for pointing this out. If we manage to hugely scale juju using mongodb
I will be very happy. I still think we should do some measurements to
convince us that we actually have some hope of doing so though.
My own measurements left me less than convinced of the
possibility, although it's been a while since I did them.

  cheers,
    rog.