Send Juju logs to different database?

[John Meinel]

I'll also note that the issues with stale reads mentioned in the
call-me-maybe article don't really apply at all for something like logging.
They also don't really apply for how we are using Mongo today. I don't
think Mongo is the best DB for append-only (delete in bulk infrequently)
logging, but it certainly suffices for the amount of data we are processing
today. The stale-reads mentioned in the article is much more about a
 single point of data that multiple sources are trying to change
concurrently, which isn't how we interact with mongo from Juju today.
For Logging, you are only appending data, so you aren't mutating data in
place, thus there isn't really a stale read problem, as the data is either
there or not, it can't be partially out of date.
For other Juju state data, we use client-side transactions on top of Mongo
for most of our operations, and while I haven't gone through every possible
interaction, it should insulate us from this particular problem. (if you
were partway through a write and master got split brain, when we switch to
a new master we should either finish committing, or rollback transactions
that were in progress. Because every transaction requires multiple layers
of writing before it is completed, in a split brain situation the master
cannot return that the transaction is successful without coordination with
its peers.)

John
=:->


On Wed, May 6, 2015 at 1:57 AM, Menno Smits <menno.smits at canonical.com>
wrote:

> On 6 May 2015 at 06:49, Lauri Ojansivu <x at xet7.org> wrote:
>
>> Hi,
>> today at Ubuntu Developer Summit:
>> http://summit.ubuntu.com/uos-1505/meeting/22437/juju-share-and-juju-sos/
>>
>> Speakers talked about changing all logging to go to MongoDB, so I asked
>> question at IRC:
>> <xet7> Would it be possible to use some different database than MongoDB
>> for logs, because of current problems in MongoDB ?
>> https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-reads
>>
>
> It is more likely that Juju will grow the ability to send logs to external
> log services using the syslog protocol (and perhaps others). You could use
> this to log to your own log aggregator or database. This feature has been
> discussed but hasn't been planned in any detail yet (pull requests would be
> most welcome!).
>
> The changes currently being made to Juju to support logging to MongoDB
> also lay the groundwork for logging to external services. This should land
> in the coming weeks.
>
> - Menno
>
>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20150506/0f056fe3/attachment.html>