getting rid of all-machines.log

[Gabriel Samfira]

Please forgive the lengthy response :). The following are just a few 
thoughts regarding this subject.

On 14.08.2014 06:13, David Cheney wrote:
> Ian asked me to post my thoughts here.
>
> I am not in favour of applications rolling their own logs, I believe
> that applications should not know anything about their log output,
> they should just dump it all to stdout and another process should take
> care of shuttling the data, logging it, culling it, whatever.
> This is summarised here, http://12factor.net/logs
This philosophy works under the assumption that all applications that 
run as services, will only do so on linux/unix systems. Services on 
Windows have no way to capture stdout unless we use a wrapper to start 
the application as a service, capture its stdout and later on, do 
something with it. Windows services mostly stream to local event log. 
Given that we want juju to run both on linux and windows, there needs to 
be a common denominator.

The ability for juju to write its own logfile *works* across most, if 
not all platforms (as it does for other apps such as nginx, apache, 
openstack components, etc). If we want to have log handling outside of 
juju, we need to implement a separate configuration on separate 
platforms (adding more moving parts). Doing this in juju removes 
external dependencies, and gives us a unified way of handling logs 
across platforms.

I can understand how this philosophy is ok for applications that are 
meant to run on a PaaS where the application needs to adapt to its 
environment, and where the PaaS prefers to not allow the developer to 
make mistakes when it comes to saving the log of the application, but we 
are not running juju on a PaaS. Juju will only be run on a IaaS or on 
bare metal, where you are adapting the environment to your needs. What 
is good for a PaaS running apps mostly on Linux, is not necessarily good 
when you are targeting bare metal or IaaS using multiple platforms. We 
have to keep in mind that not all platforms are created equal. Some have 
limitations that others do not.
>
> I think our current system with rsyslog is working fine and there is
> no reason to remove it.
There is no rsyslog for windows (well, not free at least). Also, 
installing a 3rd party service that needs to be updated separately from 
the standard windows workflow opens up the system to vulnerabilities if 
not done right.

Windows aside, on non-bootstrap nodes we currently only use rsyslog to 
tail the logs generated by juju (and written by an append redirect from 
upstart), and then stream those logs to the a state machine. So you have 
3 components that have to work together just for the end result of 
having a log streamed to a state machine. IMO there are 2 moving parts 
too many.

Most languages have a syslog package that can stream directly to syslog. 
IMHO, the rsyslog forwarder is one external dependency that should be 
easily replaced. The less moving parts, the better. This also allows 
juju to stream logs from all platforms directly to syslog (its just a 
TCP/UDP connection in the end...).

This is just my 2 cents. Its up to you guys in the end. As far as 
windows is concerned, as long as I have clear guidelines defined, I can 
adapt the PRs.

An option would be to start writing a wrapper for Windows that will be 
bundled along side jujud.exe...but it would be a platform specific 
thing. This is something I would rather not do if I have a choice.

Cheers,
Gabriel
>
> The problems with all-machines.log being to large is independent of
> log rolling or any of these other arguments. We simply log too much.
> There would be no request for log rolling it all-machines.log
> contained only useful data.
>
> Dave
>
> On Sat, Aug 9, 2014 at 2:58 AM, Nate Finch <nate.finch at canonical.com> wrote:
>> So, rsyslog rotation works fine on Linux, but we can't do that on Windows.
>> If we have to do something different for Windows, I'd rather just do one
>> thing which is cross platform compatible for all our OSes, and not have to
>> support a different configuration for each OS.  Doing it all in-application
>> also insulates us from external dependencies... if some future or past
>> Ubuntu series (or CentOS) has a different version of rsyslog, it could
>> behave differently / require a different configuration, etc.
>>
>>
>>
>> On Fri, Aug 8, 2014 at 12:40 PM, David Britton <david.britton at canonical.com>
>> wrote:
>>> On Fri, Aug 08, 2014 at 12:03:21PM -0400, Nate Finch wrote:
>>> [...]
>>>> remote syslog and to the local file log, we wouldn't need to worry about
>>>> log rotation of the local log screwing up what gets sent to the remote
>>> Do the standard rsyslog log rotation mechanisms not function well?
>>>
>>> On Windows, what about the event log (which has remote
>>> viewing/aggregation capabilities built in)?
>>>
>>> --
>>> David Britton <david.britton at canonical.com>
>>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>