rm -fr not deleting stuff

[Karl Auer]

Got a weird one here.

Not entirely an Ubuntu question, but it is Linux. I have a Synology NAS,
and I wanted to delete some old backup directories off it. I create
backup directories using rsync, hard-linking to unchanged files in
previous backups. This reduces the size of each backup by re-using an
existing copy if a file hasn't changed. Read the man page for rsync and
look at the --link-dest option.

Anyway, when I tried removing a backup using rm -fr dirname it deleted
heaps of files out of the directory and its many subdirectories, but
also issued lots of messages about being unable to delete directories
because they were not empty.

I tries rsyncing (with --delete) an empty directory into the directory I
wanted to delete, and rsync complained about being unable to remove
non-empty directories as well. Oddly though, it seemed to be able to
delete *some* of the directories that rm had not been able to remove. On
a hunch I re-ran rm -rf on the directory, and it was 8also* able to
remove some more directories, that botyh it and rsync had previously
refused to delete! So I just ran rm -fr four or five times more and
eventually it deleted everything including the topmost directory that I
was trying to remove.

So I tried on the next backup directory that I wanted to remove, this
time just using rsync, and sure enough the same trick worked - I had to
run rsync about five times, but eventually it removed everything.

My googling suggests that a corrupted disk can have files that rm won't
remove, but this disk is just fine. Nothing in the log files about
errors, fsck says the disk is good, SMART says the disk is good.

So I'm very puzzled. What would cause a directory to be not removed the
first time through, but removed the second time through?

Then I had a panic attack - what if it was the hard links? Maybe each
time I did a rm, another link was removed? That would mean I had not
just deleted the oldest backups, I had also deleted files I wanted to
keep, that were hard-linked to in younger backups! So I checked - but
even ancient files in the younger backups had been preserved - files
that would certainly have been present in the older backups. Phew!

But I am mystified. Any ideas, people?

Regards, K.

-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Karl Auer (kauer at biplane.com.au)
http://www.biplane.com.au/kauer
http://twitter.com/kauer389

GPG fingerprint: 3C41 82BE A9E7 99A1 B931 5AE7 7638 0147 2C3C 2AC4
Old fingerprint: EC67 61E2 C2F6 EB55 884B E129 072B 0AF0 72AA 9882