[Bug 1442674] Re: sysstat log corruption, time isn't changing

[Bill Cole]

Note that the "-L" flag is hardcoded in the sa1 calls of sadc, so
whatever locking that does is not fixing the issue. Seems like a
possible upstream bug?

I've had this issue in sysstat data files generated on machines ( n=9
out of 48 sa?? files from 3 different machines) where the sysstat
resolution has been changed to every 2 minutes by changing this line in
/etc/cron.d/sysstat:

     5-55/10 * * * * root command -v debian-sa1 > /dev/null && debian-
sa1 1 1

To:

    */2 * * * * root command -v debian-sa1 > /dev/null && debian-sa1 1 1

I've been able to salvage the files by looking for repeating binary
patterns to figure out the size and offsets of sadc records, finding the
one which is a runt, and rebuilding the file by splicing together the
sections before and after it, making sure the resulting file is
identical in size to the other full-day files that are parseable. This
is not an easily documented process, as it requires eyeballing hexdump
output and making educated guesses, but there are useful tips:

1. With "-S XALL" in SADC_OPTIONS, each record is 8-9KB, and for some reason they seem to alternate between 2 sizes (!) 16 bytes different, e.g. 8528 bytes & 8544 bytes. 
2. There is a header in the file between 850-900 bytes, so a good place to start looking for patterns at the end of records (useful!) is ~9K. 
3. EVERY time I've had this problem, the 2nd record has been a runt, 500-1500 bytes shorter than the normal records. 

Because excising the runt record yields what seems to be a perfectly
good file, my guess is that the root cause is collision between the
23:59 run and the 00:00 run, with the sadc -L flag for some reason
failing to do its job.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to sysstat in Ubuntu.
https://bugs.launchpad.net/bugs/1442674

Title:
  sysstat log corruption, time isn't changing

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/sysstat/+bug/1442674/+subscriptions