Backing up to DVD

Dave M DaveM at Mich.Com
Wed Feb 7 01:43:39 UTC 2007


Since I started using Ubuntu last year, I have been hacking away, trying to 
figure out how to do system, user and daily backups. To backup my system 
directories I decided to use mondo. Easy enough, just install it from the 
repositories using Synaptic and it works.

But backing up my user files was another matter. My system usage is too 
complex and I have too many user files to just throw in a DVD and burn a 
bunch of directories to it. K3b works great for that. But I need something 
that can easily do daily incremental backups, verify that the DVD was 
burned properly and can be scheduled to run automatically, unattended. I 
also need a high degree of certainty that the files actually get backed up 
and that the burned DVD's are good.

In the Windows world I was using Backup4all. The closest thing I could find 
in the Linux world was KDar. KDar is an easy to use GUI front-end for Dar. 
Unfortunately KDar and Dar do not know about DVD's.

So I needed some additional utilities and a script to tie them together. 
What I thought was going
to be a simple exercise turned out to be quite complex. Here is an example 
(see attached) of the
BASH script that I have been using to deal with burning Dar slices to my 
DVD's. This script
workd equally well using KDar, Dar with cron or Dar launched from the Gnome 
menu).

This script does the following:
o Has an option to make a redundant file for each Dar slice
   - using Par2, slow but small files (-rP)
   - using a second copy of the slice, fast but large files (-rC)
   - using both a second copy and a Par2 file, most redundant (-rB)
   - redundant file creation can be disabled, fastest burn (-rN)
o Burns the slice to the DVD
   - makes a new file system if the DVD is blank
   - adds a new session if the DVD already has files on it
   - if the DVD was blank it adds par2, dares-qt, dar_cp and dar_static.
     these are small files that could aid in a bare metal restore or with 
damaged DVD's
   - adds the log file in case you need it while restoring (if something 
went wrong)
o Has an option to verify the burned DVD
   - verify the slice files that where just burned, return fail if verify 
fails (-vY)
   - re-burn the slice if the verify fails, retry three times before 
failing (-vR)
   - verify may be disabled (-vN)
o Extracts the catalog from those slices that have catalogs.
   The extracted catalog is saved to be used as a basis for future
   differential backups.
o Deletes each slice after burning and verification to make
   room for the next slice.

I use the KDar GUI to define my backup job. KDar can run dar and use 
KDar_BurnSlice
(this script) to burn the backup slices to a DVD. Or KDar can be used to 
make a
BASH script that contains a Dar command and then that script can then be 
used to
launch Dar. This is useful in the case where you would like to have cron 
launch
the backup on a scheduled basis or have a one-click menu option to launch your
backup.

Getting started -

I am using Ubuntu and installed my packages using the Synaptic package 
manager.
I installed the following:
  - Dar
  - KDar
  - Par2
  - gzip
  - mkisofs
  - DVD+RW-Tools (growisofs)
  - Gnome-Schedule

I placed KDar_BurnSlice.sh (this script) in ~/Scripts/
Check to be sure that the script has execute privileges.

I made a working directory for my backups, catalogs, logs etc.
I use ~/Backup/ and create a sub-directory for each job.
For example, ~/Backup/HomeFull/, ~/Backup/HomeDiff/ etc.

I also made a directory to save my KDar profiles and KDar scripts.
I use ~/KDar/

I run KDar and click the "Create" button.

I click through each step to select the directories I wish to back up,
the directories to exclude, the file types to exclude and the
file types that should not be compressed etc. I use the following settings:

  - I use the date and time as part of my archive name. For example:
      HomeDiff_070205-1730
    if you plan to export the Dar command script (and launch it with cron
    or from the menu) then you must add a date command to your archive
    name so the date changes automatically each time the script is run.
    For example:
      HomeDiff_"$(date +%y%m%d-%H%M)"
    Be sure to include the quotes. Note that this will not work if KDar
    is used to launch Dar. If you use KDar to run the backup job you must
    change the date and time manually first.

  - My root directory (for the backup job) is set to /home

  - If I am doing a differential backup, I must point KDar to a
    catalog as my reference point. This can be the catalog from
    my last full backup, for example:
      /home/MyUserName/Backup/HomeFull/Cat/HomeFull_070101
    or from whatever differential point I choose to use:
      /home/MyUserName/Backup/HomeDiff/Cat/HomeDiff_070125-1250
    I would suggest doing a full backup first.
    Then do your first series of Diffs based on the catalog from the Full 
backup.
    Later Diffs can reference the catalog from an earlier Diff.

-  As each series of Diff's fills up a DVD (that usually takes me about a 
month),
    I change the DVD, do another diff and then I change the reference 
catalog (in KDar)
    to my most recent diff catalog. Be sure to export the script and save 
the profile
    after changing the reference catalog. The reasoning behind this 
procedure is that
    each DVD will then contain all of the files from the previous set of
    Diff's as well as any new files that changed while I was using that DVD.
    So each time I change DVD's I will have two copies of my files on two 
DVD's.
    One copy is the last diff on the old DVD and the other is the first 
diff on
    the new DVD. Just in case a DVD is lost or damaged.

-  I use KDar "ConfigureCreationOptions" to set up the backup job.
    o General-ArchiveStorageDir: /home/MyUserName/Backup/HomeDiff/
    o General-Logging: Full to /home/MyUserName/KDar/KDar.log
    o Slicing-SplitArchive-SliceSize: 400Mb if you plan on using parity,
      440Mb if not. Since there is a problem with growisofs adding sessions
      to a DVD when the DVD has 4Gb or more already burned, this will allow
      you to fully fill a DVD (with 10 slices) when doing large backup jobs.
    o Slicing-CommandToRun this is where you call the script:
      /home/MyUserName/Scripts/KDar_BurnSlice.sh %p %b %n %e -rP -vY >> 
%p/%b.log 2>&1
        You can choose the following options for verify and redundancy:
        -vN (Verify No)
        -vY (verify Yes)
        -vR (Verify and re-burn - default)
        -rN (redundancy No)
        -rP (redundancy using a parity file, Small files but slow - default)
        -rC (redundant copy of file, Fast but large files)
        -rB (Both parity and copy, most redundant)
    o Compression-Exclude, I exclude the following since they are already 
compressed:
      *.*_,*.aac,*.avi,*.bin,*.bz2,*.cab,*.dar,*._dd,*.deb,*.dv,*.exe,
      *.gif,*.gz,*.iso,*.jpg,*.lzo,*.m4a,*.m4v,*.mov,*.mp3,*.mp4,*.mpg,*.oma,
      *.ogg,*.par,*.par2,*.pbm,*.pdf,*.png,*.pnm,*.rar,*.rm,*.rm1,*.rpm,
      *.tgz,*.vmdk,*.wmv,*.Z,*.zip
    o I dont use Cryptography since I probably couldn't find my key if I did.
      Of course using this option will depend on your needs.
    o I exclude Directories and files as needed. In general I exclude all 
of my media
      files since I back them up separately. So this option will depend on 
your needs.
    o I do not disable or ignore any of the file system options.
      But this option will depend on your needs too.

- Be sure to save the profile that you have just created.
   Beware, if you click "Finish", KDar will run the backup job!

- If you plan on using cron or the menu to run a daily backup, use KDars
   "SaveProfile-ExportDarCommand" button. Before exporting the Dar command
   be sure that you have added "$(date +%y%m%d-%H%M)" (including the quotes)
   to the end of your archive name. If you dont, the name of the diff will be
   the same each time it is run and the older files (on your DVD) will be 
hidden
   by the new session. If your job produces more than one slice, you may 
end up with
   slices that have the same name from various backup runs. Which will not 
be useable.

- I save my profiles and dar scripts to my ~/KDar/ directory. I name the 
scripts
   HomeFull.sh and HomeDiff.sh

To set up an automatic daily backup, I use Gnome-Schedule, which is a GUI 
for cron,
to launch the script. I set up a Recurrent Scheduled Task. I used the 
'Advanced'
tab to set the schedule. I un-checked the 'No Output' box and used the 
following
command to launch the backup job:

   /home/MyUserName/KDar/HomeDiff.sh >> /home/MyUserName/KDar/Daily.log 2>&1

I also set up a menu entry for my backup so I can run it whenever I have 
something
importaint to save. I used the Alacarte menu editor to add the following 
command
to my menu.

   gksudo /home/MyUserName/KDar/HomeDiff.sh >> /home/MyUserName/KDar/Daily.log

Note that I needed to use gksudo to run the job as root (if it needs access 
to all of
the files in /home).

If you prefer not to use KDar, here is an example of a Dar command that 
uses this script:

   $dar -v -c HomeDiff_"$(date +%y%m%d-%H%M)" -R "/home/" -A 
"HomeDiff_070125-1250" -s 419430400 -D -y -E "KDar_BurnSlice.sh %p %b %n 
%e >> %p/%b.log 2>&1"

I am releasing this under the GPL 3 draft (or later) copyright. So feel 
free to
share, modify fix etc. If you find problems or have suggestions, please let me
know at "Davem at Mich dot Com" with "KDar_BurnSlice" in the subject line.
If anyone has an idea as to where I should host this script, or would like 
to volunteer
to host this script, please let me know.

Maybe the KDar developers could take a look at KDar_BurnSlice and add its 
functionality to KDar. It would be nice if KDar could burn CD's and DVD's 
without the use of scripting. 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: KDar_BurnSlice_070206.tar.gz
Type: application/octet-stream
Size: 9237 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-users/attachments/20070206/1e63bb49/attachment.obj>
-------------- next part --------------
-------------------------------
Dave M
Davem (at) Mich (dot) Com
Ann Arbor, Mich. USA

Inter-networked computers give you freedom.
The "Trusted Computing Platform" could take that freedom away.
http://www.eff.org/Infrastructure/trusted_computing/20031001_tc.php


More information about the ubuntu-users mailing list