Backing up to an external FAT32 disk

[Nils Kassube]

OK, this thread is very old, but I just had the unfortunate opportunity to 
recover data from a broken backup file which was created according to my 
suggestion below.

Rashkae wrote:
> Nils Kassube wrote:
> > Liam Proven wrote:
> >> I am trying to backup a 98% full RAID array (some 100GB of stuff)
> >> onto a 400GB FAT32 USB2 external hard drive. I don't have space on
> >> the RAID itself to create the archive, nor on my 3G root FS.
> >>
> >> I tried (in my /media directory, the mountpoint for both the RAID
> >> and the USB drive):
> >>
> >> tar -cvf usbdrive/raid.tar raid/
> >>
> >> This worked but I forgot one detail. FAT32 has a max file size of
> >> about 4G. So when the archive got to 4G, it barfed.
> >
> > Try this:
> >
> > tar c raid | split -d --suffix-length=3 --bytes=1000m -
> > usbdrive/raid_
> >
> > That will give you files of 1000MB size with filenames
> > usbdrive/raid_<nnn> with <nnn> being numbers counting from 001.
> >
> > You can test the archive with:
> >
> > cat usbdrive/raid_* | tar tv
> >
> > And you can restore a file with:
> >
> > cat usbdrive/raid_* | tar x <filename>
> >
> >
> > Nils
>
> This is the right approach, but I would use a command more like:
>
> tar -cz raid | split -b 2000m - /media/usbdrive/raid.tar.gz.
>
> And as already mentioned, to restore:
>
> cat /media/usbdrive/raid.tar.gz.* | tar -xz   (run this in the location
> where you want the files restored.)

This extra compression is a bad idea, and I want to explain why. If you 
split compressed data, you can only recover data from the beginning of 
the entire set of data up to the position where an error occurs (missing 
part or I/O error). If you have an uncompressed tar file split into 
several parts, you can recover at least some data from each single part 
if one of the parts is missing or corrupted. Well, I didn't know this 
until I had the necessity to recover vital data from broken parts.

In my case I had an uncompressed tar file of about 8GB which should be 
saved to DVDs. I used split to make it 3 parts. The parts were written to 
DVDs but unfortunately I didn't verify the DVDs. When it was time to 
restore the files from the DVDs, every single file had read errors 
somewhere. Using the dd command I could read the data following the 
broken blocks which resulted in 6 none consecutive parts of the original 
tar file. With the comands

tar xf part1
tar xf part2
... etc.

I could recover data from every single part, although tar complained about 
broken data at the beginning and end of most parts. Finally only very few 
files were really lost. If I had used compression, I would have lost most 
of the files.


Nils