Advice on backup method

Sun May 29 14:38:29 UTC 2016

On Sunday 29 May 2016 04:50:02 Chas IRONS wrote:

> Good day all
>
> As a retired end user of Ubuntu Linux since 2005 I cannot get an
> easy/simple backup method to work. Would File Manager copy to a flsh
> drive and restore a damaged file safely? I have never needed to
> recover but it may happen.
>
> I have used the Backup front end for deja-dup each month-end many
> times but this time it would not switch to a new flash drive. The
> Software Centre also had many bad comments. So I installed Back in
> Time with its recommended default settings but it failed with many
> pages of errors about Evolution files. My Evolution folders have 65MB
> of history about banking, medical,tax and investment transactions that
> I must keep.

The flash drive has a limited number of write cycles, so I would never 
consider that as a medium for backup.  And I played with tape, but found 
the tapes and drives in the price range I could afford, were both too 
small and of very poor quality.  I was doing it all with amanda, which I 
think, because the whole structure of the backups it does, is the best 
idea in backups that ever was proposed. First, you set it up to run 
every night at say 1AM, and its configuration file tells amanda how many 
days it has to do a full backup. So this allowed me to backup what had 
become nearly 20 gigabytes by setting it for 6 days per "cycle", and 
dividing the disklist entries up so most of the individual files were 
often under a  gigabyte.  By that means I was able to let amanda adjust 
the schedule so that on any given night, it came within 100 megs of 
filling a DDS-2 tape.  But the tape, and the drives weren't all that 
usable in the longer view, and tape errors by the time the drive was 6 
months old, and that particular tape used perhaps 7 times, so I wasn't 
getting the backups I needed.  Software was not the problem, crappy 
hardware was.

About that time, one of the amanda developers added the ability to use 
virtual tapes on hard drives, which had progressed to the point where a 
500Gb drive was the "commodity" drive.  So I bought a drive, half the 
price of a tape drive, and reconfigured amanda to use it.  That was a 
decade and change back up the log.  It Just Worked(TM) and now, with 
smartctl monitoring drive health, it has warned me of impending drive 
failure far enough in advance that I could go to town and buy another 
drive to replace either the drive amanda was using, or the main drive. 
Recovery even to a bare new main drive with nothing on it but a fresh 
install of the flavor of linux you were using is nearly automatic if you 
are using my amanda "wrappers" which put a copy of amanda's housekeeping 
database at the end of each "tape" backup session.

And the recoveries are done in 5% of the time it would take using tapes 
because while the tape is a sequential medium that must be read from the 
beginning to locate a file each time, the virtual tape on a hard drive 
is random access, taking a few milliseconds to do what might take an 
hour or more from a tape.  Whats not to like?

Virtual tapes, using amanda, and my wrapper scripts to give you a nearly 
bare metal recovery in the event of a drive meltdown, have just worked 
for over a decade now, with no data loss I didn't decide to throw away 
myself.  The wrapper scripts are available from my web page in the sig.
Bash scripts, they may need a tweak to optimize them for your system.
amanda will backup its own configs and tape database of course, but that 
is as they existed, a day old.  My scripts do a specific backup of both 
as they exist after amanda has completed the run, so that then is 
exactly what amanda just recorded. 100% up to date, not a day old.

I'm still on wheezy here for 4 machines because one app that I use on 
those machines has not been updated to run on jessie yet & one lappy 
running xubuntu that has nothing on it, its used as a terminal to write 
gcode on when its in use, or I might take it on the road if I get called 
to go put out an engineering fire at a broadcast facility, but now at 81 
yo with advanced RA,I am not doing anything but the local radio station. 
As a broadcast engineer, I am one of those that actually fixes it when 
the transmitter upchucks.

All 4 of these machines, since they are accessable on my local network, 
are backed up nightly.  amanda actually stands as the acronym for 
Advanced Maryland Automatic Network Disk Archiver, growing out of the 
system Maryland U. wrote for its unix machines back in the 1980's.  Its 
available from every distributions repository's via its package 
managers. But, because rpm doesn't have the abilities to switch write 
perms as required to do a proper install, I would build it from scratch 
if you are useing an rpm based system.

So just because I can, I am running version 3.3.7p1 on this machine as 
server, and 3.3.1-4, from the wheezy repo's on the "clients".

> The first few errors in the log below were followed by 10 pages of
> Evolution errors: Please advise me!
> [E] Error: rsync: symlink

symlink's are always owned by root.  Thats another place where amanda 
stands out, in that if properly configured, it uses just enough rights 
to get that job done, and no more.

It looks as if that is the primary cause of all your error posted here.

So forget your preconceived ideas that you absolutely have to back it all 
up on one swell foop on Fridays, by doing only those files which have 
been touched every night, amanda has your back covered and can restore 
to the condition it saved when it ran last night if the main drive 
fails, or will via smartctl, tell you in advance when the drive its 
using is about to upchuck. But you have to forget the full backup 
concept, amanda will do them according to its internal schedule, but 
always within the number of days you give it to do that full, the 
incrementals it does on the other nights still gives you the equ of a 
full backup every night, using 1/3rd of the media space used for a full 
in one pass on Friday's. And because that hard drive is 1000x more 
dependable than tapes, and usually faster at writes, and 20000x faster 
at a recovery, after nearly 20 years of using it, amanda itself has not 
lost a single byte I needed later.  I simply would not consider any 
other method.

> "/media/chas/BCA6-796F/backintime/Charles-PC/chas/1/new_snapshot/backu
>p/home/chas/.config/opera/SingletonCookie" -> "11075476278167463458"
> failed: Operation not permitted (1) [E] Error: rsync: symlink
> "/media/chas/BCA6-796F/backintime/Charles-PC/chas/1/new_snapshot/backu
>p/home/chas/.config/opera/SingletonLock" -> "Charles-PC-3224" failed:
> Operation not permitted (1) [E] Error: rsync: symlink
> "/media/chas/BCA6-796F/backintime/Charles-PC/chas/1/new_snapshot/backu
>p/home/chas/.config/opera/SingletonSocket" ->
> "/tmp/.org.chromium.Chromium.y4TlhJ/SingletonSocket" failed: Operation
> not permitted (1) [E] Error: rsync: symlink
> "/media/chas/BCA6-796F/backintime/Charles-PC/chas/1/new_snapshot/backu
>p/home/chas/.config/pulse/1b13e7b811a0e3d43e110330544bc30b-runtime" ->
> "/tmp/pulse-PKdhtXMmr18n" failed: Operation not permitted (1) [E]
> Error: rsync: mkstemp
> "/media/chas/BCA6-796F/backintime/Charles-PC/chas/1/new_snapshot/backu
>p/home/chas/.config/evolution/addressbook/views/.current_view-file:___h
>ome_chas_.local_share_evolution_addressbook_local_1148126609.11372.6 at ub
>untu.xml.nXKWlY" failed: Invalid argument (22)

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>