Make systemd journal persistent | remove rsyslog (by default)

[Xen]

Jamie Strandboge schreef op 12-01-2017 17:20:

> Why not just do '1' and let rsyslog remain? The standard logs are 
> rotated so
> this shouldn't be overly burdensome. Have you measured how much the 
> duplicate
> logs would take on a typical system?

Personally I think the suggestion to remove rsyslog is just change for 
the sake of change.

It is another thing people have to "catch up with" for little benefit. 
Actually, perhaps none.

The binary journald format just makes things less accessible. If there 
was a standard independent tool to read it, maybe fine. But it is clear 
the ordinary person scripting away can just do less with it. Using 
journalctl as a user is cumbersome and I often just grep it through 
"grep ." just so I get long lines. I mean journald really, of which 
journalctl is the front-end of course.

Being able to just tail whatever log you want is just amazingly powerful 
from a simplistic point of view that is accessible to just about 
everyone. Using a different command that requires a switch to select the 
appropiate log (if any) just is not.

The non-persistent journald log is a detriment I guess if you want to 
diagnose shutdown issues. Other than that there is not really much point 
to it for a regular system. Perhaps an easier way is to make it more 
accessible to turn the persistent log on (creating a directory is not 
very intuitive in that sense) or for the log to be not fully persistent 
but only for say the last 2 or 3 reboots.

If you could enable a 3-boot systemd log (journald log) including a 
mechanism to "splice" individual logs into actual log files by default, 
I think much less people would have a problem with journald and it would 
not replace the textual log format completely. What you would want, I 
guess, is for the old formats to ALSO be accessible This redundancy in 
that sense is not a problem, even if the rotated .gz format is 
cumbersome to work with. So if you will make journald persistent, I 
would suggest:

- make it persistent for 3 boots
- if applications (including plain syslog) log to journald enable 
automatic default appending to actual existing individual textual log 
files like of old.

Then you would have something that for most people would be a reasonable 
and acceptable rsyslog replacement because it means it becomes a 
stepping stone for something _more_ instead of having _less_: you would 
have the same features for the ordinary user as well as a more solid 
"building block" to build it on, being that binary format with its 
advanced features. Then you have the best of both worlds, and regular 
people do not have to turn the persistent log on, but they also do not 
fill up the disk with old data people might consider sensitive, or they 
just don't want to have it around, or it just doesn't have any use.

I mean there are certainly improvements to be had over plain (gzipped) 
text files, the simple task of grepping the files requires different 
commands for the .gz files and the regular ones, to begin with. That's 
just very annoying. In this way perhaps systemd (journald) could 
simulate the old rotation, or just keep a limited number of textual 
files on disk. It is a bit of a shame that ext4 is not so great at 
supporting compression on the fly and the only means for having it are 
btrfs and zfs, which makes it inaccessible for a lot of people and not a 
great default way of doing things. I mean in the sense of advancing a 
textual storage format on disk that doesn't take up as much space but 
which would be an improvement over the gz format.

I think the ideal thing is to just for now, or maybe at all, in any 
case, to just simulate the old rsyslog behaviour with the rotation. Then 
no one but the most advanced loggers out there will even see the 
difference. The migration to journald will be transparent to them.

Of course this requires cooperation with the systemd people but that is 
what any real solution in the end would look like, I guess. I presume.

For me to have in particular logs of other applications isolated in a 
binary format is a great detriment and doesn't make the system more user 
friendly. Systemd has a bridge to deal with SysV init scripts, why not 
also a bridge to the text-only format?

Just keep them duplicate on disk, for most people it won't matter, and 
for those for which it does, I think these people could be making an 
informed choice. But this way by default things keep working as they do, 
while at the same time building a more solid foundation of journald as 
the core of it.

In the meantime I just think you shouldn't change anything if it is not 
going to be an improvement.

There is no point in migrating if it doesn't improve things and for many 
people they keep running after changes that have little benefit to them.

Ubuntu releases every six months (and it derivatives) and I believe that 
apart from the welcome of the new kernels and the new HWE the pace is 
too rapid for many people to be able to cope with system upgrades, 
changes, stuff that stops working, etc.

Maybe in the beginning it was useful but I am just going to be so bold 
and rude to say that perhaps we have entered an era where 
/solidification/ or /consolidation/ is now more important and a release 
schedule of 8 months would now be more appropriate.

I feel as though in particular some derivatives keep running after 
changes and releasing new versions without having time to sit back and 
relax and to see what they've done. To look back on the work done, is 
what I mean here.

It's too fast. I'm still on 16.04 myself and we're already approaching 
17.04. 16.10 jumped two major kernel versions (4.4 -> 4.6 -> 4.8) and 
most will never have been running 4.6 and in 4 days it is coming to 
16.04 as well as point release 2 I read somewhere that I didn't want to 
read it on.

The feeling starts to be that of too-fast and too-rapid advances, also 
in the kernel. At bit like the improvements in Wordpress when they want 
from version 3 to version 4 seemed rather... unnecessary and much like 
feature growth apart from real solid advancement and structural 
improvement. Well that is irrrelevant here of course.

I am just saying for me personally everything goes too fast. I had no 
problem with the pace when 14.10 -> 15.04 -> 15.10 -> 16.04 was current. 
Nowadays for no real difference in my situation what that goes, I feel 
there is little point in continually changing my systems. But for 
developers, the rapid pace also means constant pressure.

Well sorry for this long 'rant' like message. I was not meaning to write 
so much or so diverse.

And I guess I was a bit hasty with my earlier response.

But.

I think, during this era, you should hold back on changing for the sake 
of change.

Unless you solidify the new solutions, and even if you do, it will mean 
that people who like a little stability may need to jump from 16.04 to 
20.04 or something in one go. Because in the meantime, or in between, 
all these "advances" happen that are not really advances because they 
take away important stuff by replacing it with very little.

So take care not to erode what you have as you try to make journald or 
systemd more prominent and more of a solid foundation. You must not 
replace your house with a shack just because the shack is made of 
superior materials.

Ensure you can build a real house out of those superior materials before 
you decide to abandon the old, or before you decide to move house.

And one way to do that in this case is to ensure that the "superior" 
(usable) rsyslog format remains accessible as you improve the innards of 
the system. That's all I can say here.

Regards.