Should I split a big (8TB) backup disk into more than one partition, or not?

[Chris Green]

On Sun, Sep 20, 2020 at 08:45:22AM +1000, Karl Auer wrote:
> On Sat, 2020-09-19 at 07:07 -0400, Charles M wrote:
> > Hoping now that 18TB drives are a thing that the price for 8TB will
> > go down again and we'll have a backup for our backup.
> 
> I suggest everyone read up on "mean time between failures" and then,
> even more importantly, read up on "bit error rates".
> 
> A "bit error" is where a bit that was written as 1 (or 0) is
> subsequently read as 0 (or 1). The more data a disk (or set of disks)
> stores, the more likely a bit error becomes. BER is expressed as the
> number of errors that can be expected per number of bits written.
> Modern disks have ludicrously low BERs (yay!)- but ludicrously high
> storage capacities (uh-oh).
> 
> > I filled my 8TB in 7 months and I'm glad I didn't partition it
> > further. I compressed the data and saved a tonne of space.
> 
> Apropos compression, a bit error on a compressed or encrypted disk or
> file is generally a much bigger deal, because it can render all the
> data on the disk or in the file unreadable.
> 
> So use error correction (like RAID) or have backups. Multiple backups.
> 
I don't compress or encrypt my backups.  I run incremental backups of
all important files at hourly intervals to a separate drive on my
desktop machine and at daily intervals to a remote machine (which is
where the new 8TB disk is going).

Thus I have backups that look exactly like my home directory (less
unimportant stuff in directories called tmp, cache and things like
that) at hourly intervals on my desktop and daily intervals on the
remote backup machine.  As they get older they get weeded out. E.g. I
have on my desktop machine:-

cur    day-2  day-4  day-6  hour-1   hour-2  hour-4  hour-6  hour-8 week-1  week-3  week-5
day-1  day-3  day-5  day-7  hour-10  hour-3  hour-5  hour-7  hour-9 week-2  week-4

The daily backups on the remote machine are less efficiently weeded
out, there should be one per month for old backups but in reality not
everything gets 'thinned out'.  I'm happy to keep a few extra ones. I
have backups there back to January 2015.

The 3Tb disk on the backup machine is about 50% full now which is why
I have got the new 8TB drive.  My home directory, as backed up, is
around 160Gb but the 137 backups I have occupy less than half of the 3Tb
disk drive because they're incremental (there's other things backed up
there as well, such as my laptop).  I use rsync's ability to
create hard links to do this.

(I do also back up /etc by the way)


-- 
Chris Green