getting rid of all-machines.log

[Matt Rae]

On Thu, Aug 14, 2014 at 8:23 AM, Nate Finch <nate.finch at canonical.com>
wrote:

> The front page of 12factor.net says "offering maximum portability between
> execution environments" ... that's exactly what I'm going for.
>
> We're going to support Windows. Windows does not have rsyslog or
> logrotate.  We *have* a solution which is cross platform.
>
> My main concern is that if we use rsyslog on linux and something else on
> Windows, now we're supporting and maintaining two things, when they could
> both be the same.  Why support two things when you can support just one?
>
>
I'm sure writing to stdout and redirecting to a logfile that gets watched
> by rsyslog is the linux devop standard.  But it has also proved annoyingly
> complex for us to configure and maintain. We've wasted several developer
> *weeks* of time getting it to work right.
>
> Why would we continue to maintain the rsyslog stuff when it has been such
> trouble for us?  Just because it's the standard, doesn't mean it's the
> right tool for this specific job.
>
>
I talked with Ian about this a little this morning, and my suggestion was
> that we give users a way to turn off our in-app log rotation and let them
> set it up themselves external to the app (with rsyslog or whatever they
> like).... I just don't want to have to configure it or support it.
>

Yeah this sounds like a good suggestion. Maybe we can solve the problem in
a cross platform way but still have options to log to stdout/syslog in case
a user wants to integrate with an existing logging system themselves. A lot
of applications have options of logging to stdout or to syslog depending on
which you prefer.

Principle of least astonishment and taco bell programming [2] are a couple
ideas that might be good to keep in mind when deciding how we do it.

[1] http://en.wikipedia.org/wiki/Principle_of_least_astonishment
[2] http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html


>
>
> -Nate
>
>
>
>
>
>
> On Thu, Aug 14, 2014 at 9:04 AM, Gustavo Niemeyer <
> gustavo.niemeyer at canonical.com> wrote:
>
>> As a side note, MongoDB offers a "capped collection" mechanism with
>> the semantics that you can insert rows at will, and it rolls around
>> automatically by replacing oldest entries with the newest ones. This
>> tends to be a very convenient way to do structured logging, both on
>> the writing and on the reading side. Using that as a general rsyslog
>> might be unwise, as there are a number of details to get right and the
>> volume may get wild depending on the application, but at least for the
>> juju implementation itself it might be quite convenient. Imagine being
>> able to ask "please tell me the output of the last run of the
>> db-relation-joined hook on unit db/3".
>>
>> On Wed, Aug 6, 2014 at 11:24 AM, John Meinel <john at arbash-meinel.com>
>> wrote:
>> > all-machines.log is where we aggregate the messages from all
>> > machines/units/etc.
>> > It is likely to get big, which is why we want log rotate, but if you
>> want to
>> > be able to "juju debug-log" and actually get the feed of everything
>> that is
>> > going on, that needs to be *somewhere*. And yes, we'd like to switch to
>> > something like log stash instead, but until we get there we do still
>> need
>> > it.
>> > John
>> > =:->
>> >
>> >
>> >
>> > On Wed, Aug 6, 2014 at 4:13 PM, Tim Penhey <tim.penhey at canonical.com>
>> wrote:
>> >>
>> >> On 06/08/14 16:11, Nate Finch wrote:
>> >> > all-machines.log seems both redundant and a ticking time bomb of disk
>> >> > space usage.  Do we really need it?  Can we drop it and maybe later
>> >> > schedule some time to use something like logstash that is both more
>> >> > featureful and is cross platform compatible (unlike rsyslog)?
>> >>
>> >> not yet...
>> >>
>> >> debug-log uses all-machines.log, we cannot get rid of it at this stage.
>> >>
>> >> We can't drop it until a replacement is in place.
>> >>
>> >> Tim
>> >>
>> >>
>> >> --
>> >> Juju-dev mailing list
>> >> Juju-dev at lists.ubuntu.com
>> >> Modify settings or unsubscribe at:
>> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>> >
>> >
>> > --
>> > Juju-dev mailing list
>> > Juju-dev at lists.ubuntu.com
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>>
>> --
>> gustavo @ http://niemeyer.net
>>
>> --
>> Juju-dev mailing list
>> Juju-dev at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20140814/ececdb42/attachment.html>