How to backup before a release upgrade from Ubuntu 18.04.5 LTS server?

[Bo Berglund]

On Mon, 27 Sep 2021 22:43:01 +0200, Bo Berglund <bo.berglund at gmail.com> wrote:

>I have run some tests only involving the Download dir in my home, which includes
>some files I want to exclude.
>This uncovered that the path in the command line:
>exclude-from='home/user/rsyncexcude.txt'
>actually must be a full actual path!
>
>It worked when I ran the command from / but when I moved into my home dir and
>ran the exact same command it failed to find the file...
>It actually seems like it uses the current dir from where rsync is called as a
>base dir when trying to find the exclude-from file.
>
>I tested with this in the exclude file:
>unt*/video/*.mp4
>
>This is a test to exclude a video file in Download/untrunc-master/video/ but
>using two wildcards (which does not work on the command line with ls...).
>rsync is fine with these two wildcards.
>
>Now it is evening tea-time here...

Following tea I decided to test the rsync sommand using an exclude file.
Here is my rsyncexclude.txt file to make sure no videos would be copied and that
mnt and nfs would not be touched either:

mnt/*
nfs/*
home/*.gz
home/*/www/*.mp4
home/*/www/*.jpg
home/*/www/*.part
home/*/www/*.ts

The commands I ran (in a screen container) was started shortly after midnight:

sudo su
rsync -a --exclude-from='/home/bob/exclude.txt' / /mnt/backup/20210928

I left the screen container (Ctrl-A D)

After a bit I checked that it was running using top and found two rsync taska
each using 90-100% CPU. I expected something like a 40 GB copy on the external
drive.

Today I checked if it was done using top and it was still running after 8 hours
with the same high CPU usage. Then I also checked the disk usage of the external
drive and found that it was using 186 GB!!!!

So I re-entered the screen and stopped rsync using Ctrl-C.

What happened then was that it displayed this:

rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(644)
[sender=3.1.2]
rsync: write failed on "/mnt/backup/20210928/proc/kcore": No space left on
device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.2]


And when I looked at the backup I of course found all of the mp4 videos I did
*not* want copied...

So my exclude did not work, unfortunately. However the /mnt and /nfs dirs in the
copy were empty.

I have manually removed the mp4 file below the home/bob/ dir and then the total
for the backup is 23G, a far cry from the 186G from last night.

Now I have been trying to figure out why this happened and how my next attempt
should look like tonight.
This time I will not use an exclude file but instead put all of the excludes
onto the command line (as root):

# rsync -a --exclude 'proc/*'  --exclude 'mnt/*' --exclude 'media/*' --exclude
'nfs/*' --exclude 'lost+found/*' --exclude '*.mp4' --exclude '*.part' --exclude
'*.ts' / /mnt/backup/20210928B/

This time I hope it will exclude all mp4 videos and also exclude proc/ where the
previous run stalled as well as media/ and lost+found/.

Hopefully this will work better.


-- 
Bo Berglund
Developer in Sweden