out of space on /root

[Ken D'Ambrosio]

This is a very rudimentary attempt to strip out stuff (timestamp, PID) 
that would be largely unique per-line, and then pull out the most 
frequently recurring messages:

export LOG=/var/log/syslog
cat $LOG | perl -e 'while (<>){s/^.{16}//; s/\[[0-9]+]//;print;}' | sort 
| uniq -c | sort -n

Now, 27 GB would take a L-O-N-G time to sort through, so instead of "cat 
$LOG" I'd probably change it to something like "tail -10000 $LOG".  (I 
only bother with stuffing the logfile into $LOG so that it will fit in 
the constraints of my ASCII mail compose window; apologies.)  The output 
will look something like this:

[lots of onesy-twosy stuff here]
      84 foo spamd: prefork: child states: II
      86 foo spamd: spamd: setuid to spamd succeeded
     153 foo dovecot: imap(ken): Logged out in=94 out=987

The number in the left-hand column is the number of times that 
particular message occurred.  You could pipe everything through "tail 
-20" or somesuch, just to grab the bottom of the output, with, 
presumably, the most relevant errors/messages.

Good luck!

-Ken


On 2017-03-06 08:10, Xen wrote:
> Simos Xenitellis schreef op 06-03-2017 11:57:
> 
>>> In general commands such as:
>>> 
>>> cat "file" | tail -n 500
>>> 
>>> will get you the last 500 lines.
>> 
>> Preferably, you can write straight ahead
>> 
>> tail -100 /var/log/syslog
> 
> Yes, that's what I said. But his files are no longer in /var/log, so
> that doesn't help.
> 
>> 3. I think it is more performant not to start with "cat" because "cat"
>> would go reading through all the file
> 
> Yes, that's what I said.
> 
>> before "tail" will be able to prune all but the last 100 lines.
>> Perhaps "tail" has some optimizations to ignore reading the majority
>> of the 27GB syslog,
> 
> Yes, that's what I said.
> 
>> so it is better to to just "tail".
> 
> Yes, that's what I said ;-).