Help, my disk array has one dead member

[Xen]

Kevin O'Gorman schreef op 23-03-2017 6:14:

> I may already have enough, or nearly so.  I have copied all the files
> I could in the root directory of the raid, and captured names of the
> ones I could not read.  Of course there are some directories I could
> not read. and anything in them is lost too.  But there are some
> directories that are readable.  So I'd like to do the same thing in
> those.  There are enough of them that I'd like to automate it,
> including taking note of what was unreadable.  Filenames themselves
> will tell me a lot.

Well I guess it is easy enough to test a file for readability and do 
something with that right.

"find" would never descend into unreadable directories so you can skip 
those automatically.

If you only care .. no.

find $root -type d >/dev/null should give you an output of things that 
error on stderr.

So

find $root -type d 2>&1 >/dev/null should give you a list of erroring 
directories on stdout.

At the same time

find $root -type f | while read f; do
     cat "$f" > /dev/null && {
         # copy this file somewhere
     } || {
         echo "$f"
     }
done

would give you on stdout a list of files that either fail to be "catted" 
or fail to be copied, which would probably be the same thing so you 
could also immediately try to copy them and save a little bit of time.

and everything that fails to cat or be copied would be output so you can 
direct that to a file.

You'd end up with a directory tree of all files that could be copied 
(you copy them into the same spot in that sense) and a text file of all 
those files that weren't copied.

You will then also have a separate directory list of directories that 
could not be traversed.

You can simply combine the two lists and alphabetically sort them which 
would put the directories on top of any files that follow it.

So yeah, first command > dirlist.txt
Second command > filelist.txt

sed -i "s at .*@&/@" dirlist.txt, to give the directories ending slashes if 
you like.

cat dirlist.txt filelist.txt | sort > combinedlist.txt would give you 
the combined sorted list.



> I'm not going to image the broken RAID, because I'm just not going to
> spend the time dealing with tiny fragments.

That was not really the intent though, it was just a precautionary step 
that was advised in that sense, but I guess you mean using photorec on 
it.

That would create a mass of files that would be completely undoable.

Linux tends to litter a lot of copies of small files all around, is my 
experience :p.

Every file you've ever worked on probably has at least 6 copies lying 
around :p.


> I'm going to recover what
> complete files I can, though I'm not sure any data files will be
> useful, because some scripts may be recovered and it's my hobby and
> I'll feel better about it if I go that far.  I have a working tar file
> of the main sqlite database as of 3 days ago and the only other stuff
> that really matters is the little scripts I use to automate some of
> the chores.  I don't want fragments that I'm not sure were current --
> I'd rather rewrite from memory or from scratch.  Then I have a day or
> so of work to redo, and I have a log book that tells me what that was.
>  I should do what recovery I can, use the new disks to replace that
> raid entirely, and order a new set.  When I'm back up and running I'll
> test the bejabbers out of the raid's drives before I put any back in
> service.

Create a backup md array to go with it if you care :p. I guess that 
would create 3 mirrors :p. But I guess in your case it would be raid 5 
then.

The above won't copy empty directories.

> And background: I've lost data before, like my PhD research back
> around 1999.  Eeek!.  So twice I've gone to data recovery services and
> paid a few thousand bucks to get data back.  Because of this, nowadays
> I stay pretty current with backups of crucial stuff.  But my raid is
> so big I just have to take my chances.  I didn't have a great
> destination for storing backups.  This is changing, and I now have
> another big machine, on which I'm installing a RAID that will mirror
> the things on this one.  But I'm going to use the drives I got for
> that as the replacement RAID in the old machine.

Oh right, yeah.