brainstorming for UDS-N - Performance - disk footprint

[David Henningsson]

On 2010-11-01 14:33, Steve Langasek wrote:
> Hi David,
>
> On Fri, Oct 15, 2010 at 10:36:21AM +0200, David Henningsson wrote:
>>>> Turns out that rsyslog writes most entries three times (!), e g if you
>>>> have a message coming from the kernel, it shows up in /var/log/syslog,
>>>> /var/log/messages, /var/log/kern.log and /var/log/dmesg!
>>>> I think this is a waste of SSD life, disk space, CPUs will go hot, which
>>>> will in turn warm up the oceans, causing all Narwhals to die! Oh no! ;-)
>>>> Back to business, can we just write everything to /var/log/syslog and
>>>> drop everything else? My assumption is that most users doesn't use these
>>>> files at all, and those who do, are normally aware of the "grep" tool.
>>>> The slightly more advanced users can configure rsyslog to fit their needs.
>>>> Now, I don't know if this is something for an UDS session, a wishlist
>>>> bug on LP, or simply a discussion here, so feel free to redirect me to a
>>>> better forum if necessary.
>>> I'll note that these files are typically small and don't grow much
>>> during a session, except when something goes wrong.  Then they can get
>>> huge, making things even worse.
>> Finally, a reply :-)
>> I guess this isn't not only about disk footprint but also about SSD
>> life, I/O bandwidth, I/O time, CPU time etc. Especially in constrained
>> environments (think netbook, ARM, etc).
>> For me it's just confusing with all these logfiles with duplicated
>> information. I think they make Ubuntu more bloated and less "light",
>> without giving anything of much value in return.
>> But what do you folks say, should I just comment most of rsyslog.conf
>> out, provide a debdiff, attach to a LP bug and hope for sponsorship, or
>> is this something requiring more discussion?
> Not sure if you had a chance to discuss this last week at UDS,
Actually, it was discussed in an UDS session related to "fitting Ubuntu 
on a CD", and Martin Pitt brought it up. I believe we decided to go with 
keeping /var/log/syslog and drop everything else. The reasoning was that 
it's good to have some place where everything was logged in order (so 
you can see the sequence of events), and it's easy to grep if you want 
something specific.

(Reference: line 161 in the gobby document 
performance-desktop-n-install-footprint.)

> but I'll
> chime in here to say I think it's important to have a spec for this and not
> just change things by way of a bug report.  Questions of "why is logging the
> way it is" need to have clear answers documented centrally as part of our
> platform documentation, not just a trail of changes recorded in a changelog.
> Otherwise it becomes impossible later to distinguish between implementation
> bugs and disagreements about intent.
Seems fair to me.
> For the actual implementation, I propose this strawman:
>
>   /var/log/auth.log: synchronous logging of anything related to
>     authentication; synchronous so that this maintains its current utility as
>     a potential audit log in the event of system crashes triggered by an
>     attacker.  If this is causing too many disk writes, this ought to be
>     fixed elsewhere - e.g., cron logging every 5 minutes ->  turn the cron job
>     into a daemon; too many logs of ssh login attempts ->  implement better
>     adaptive firewalling.
>
>   /var/log/kern.log: everything kernel-related; separated out because this
>     can be quite verbose and is usually orthogonal to what we log elsewhere.
Isn't this already provided in /var/log/dmesg?
>   /var/log/mail.*, /var/log/news.*: everything mail/news-related.  These are
>     likely to be empty on systems where we care about logging performance,
>     anyway.
>
>   /var/log/daemon.log: daemon.*.  Probably no need to keep this as a separate
>     log except for historical compatibility.
>
>   /var/log/messages: *.info;auth,authpriv,cron,daemon,mail,news,kern.none.
>     Instead of *only* logging messages of priority {info,notice,warn}, log
>     *all* messages of info and above; otherwise we counterintuitively have to
>     look in /var/log/syslog (the all-in-one log) for *higher*-priority
>     messages.
>
>   /var/log/syslog: *.*;auth,authpriv.none.  Comment this log file out,
>     leaving it as an example for admins who want an all-in-one log.
>
>   /var/log/debug: comment this out as well.  I can't remember the last time
>     /var/log/debug was useful to me without having to first reconfigure a
>     service to log verbosely.  Looking at my system, it doesn't contain
>     anything useful.  (Alternatively, we could use $foo.info for all the
>     other logs and shunt all debug info here exclusively.)
>
>   /var/log/lpr.log, /var/log/user.log: drop, like cron.log has been dropped.
>     These are already logged to /var/log/messages today, which seems
>     sufficient to me.
>
> Cheers,