Can't access drive - Need raid5/linux help

[imt17]

Of course that assumes I got the full backup. I think I did and that is all I can go on.

Code: Select all

root@fvdw-sta-kirkwood:/ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md2 : active raid1 sda9[0] sdb9[2]
      875456 blocks [5/2] [U_U__]
     
md1 : active raid1 sda8[0] sdb8[2]
      843328 blocks [5/2] [U_U__]
     
md3 : active raid1 sde5[4]
      255936 blocks [5/1] [____U]
     
md4 : active raid5 sda2[0] sde2[4] sdc2[2] sdb2[1]
      3898947584 blocks super 1.0 level 5, 512k chunk, algorithm 2 [5/4] [UUU_U]
     
md0 : active raid1 sde7[3]
      16000 blocks [5/1] [___U_]
     
unused devices: <none>


Code: Select all

root@fvdw-sta-kirkwood:/ # ls -l /dev/sd[abce][5789]
brw-rw-rw-    1 root     root        8,  37 Dec  5 18:13 /dev/sda5
brw-rw-rw-    1 root     root        8,  39 Dec  5 18:13 /dev/sda7
brw-rw-rw-    1 root     root        8,  40 Dec  5 18:13 /dev/sda8
brw-rw-rw-    1 root     root        8,  41 Dec  5 18:13 /dev/sda9
brw-rw-rw-    1 root     root        8,  21 Dec  5 18:13 /dev/sdb5
brw-rw-rw-    1 root     root        8,  23 Dec  5 18:13 /dev/sdb7
brw-rw-rw-    1 root     root        8,  24 Dec  5 18:13 /dev/sdb8
brw-rw-rw-    1 root     root        8,  25 Dec  5 18:13 /dev/sdb9
brw-rw-rw-    1 root     root        8,   5 Dec  5 18:13 /dev/sdc5
brw-rw-rw-    1 root     root        8,   7 Dec  5 18:13 /dev/sdc7
brw-rw-rw-    1 root     root        8,   8 Dec  5 18:13 /dev/sdc8
brw-rw-rw-    1 root     root        8,   9 Dec  5 18:13 /dev/sdc9
brw-rw-rw-    1 root     root        8,  53 Dec  5 18:13 /dev/sde5
brw-rw-rw-    1 root     root        8,  55 Dec  5 18:13 /dev/sde7
brw-rw-rw-    1 root     root        8,  56 Dec  5 18:13 /dev/sde8
brw-rw-rw-    1 root     root        8,  57 Dec  5 18:13 /dev/sde9


When I fist went over to the house, and friend couldn't access, there were two disks that were red. I hit the power switch and it took a little while but the unit powered down. I then started the unit back up and then there was only disk red. This was Disk 3, but it was Disk 4 that I discovered had the hardware failure. So that is strange. I was able to use the web server and log in. It showed that the raid was being rebuilt. I tried attaching the WD USB drive to the USB port and perform a backup using the Lacie web server and it failed to backup at the start. We then hooked up another 3TB USB drive to his Mac and we copied the data folder over. He then went away for a week. Upon his return he "thought" the backup was done. I told him to power off the NAS before I headed over via the switch, since the last time it took a while. He also told me the disk was still red and I told him to mark the disk. When I got there 30min later the Lacie was still on. It was't powering down. I was able to pull up the web server on the static IP but it was not taking the login. Since the unit was no longer powering down, I pulled the plug. I then tried replacing disk 3, before powering back up, thinking all of the other drives were fine so then I could log back in and rebuild the raid5 if it was;t being done so automatically. When I powered the unit back up after it went through each drive blinking blue for the startup all lights were blue, which I thought was strange. Figured 3 should have been red, since the raid would have been degraded and would be re-building the raid. I then tried to log in and could not access the web server. We went back to his Mac and was unable to access the shared folder so knew something was wrong. I power down the unit via the switch and then brought it to my house and started this post.

[imt17]

forgot to add that when I got home and also check the WD Drive, that we were copying the files off the NAS to, I saw that it only copied up to a certain point and the other folders where greyed out meaning they were set to copy but never made it that far so over the course of the week he was gone, the backup copying stopped probably due to some errors on the NAS side.

[Jocko]

:scratch

So currently the device nodes in /dev are update.

their raid level are 1, so you should be able to repair them.

For testing can you try this

Code: Select all

mdadm --re-add  /dev/md0 /dev/sda7

and post

Code: Select all

mdadm --detail /dev/md0

[imt17]

Code: Select all

root@fvdw-sta-kirkwood:/ # mdadm --re-add  /dev/md0 /dev/sda7
mdadm: re-added /dev/sda7


Code: Select all

root@fvdw-sta-kirkwood:/ # mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Thu Jan  1 00:00:15 1970
     Raid Level : raid1
     Array Size : 16000 (15.63 MiB 16.38 MB)
  Used Dev Size : 16000 (15.63 MiB 16.38 MB)
   Raid Devices : 5
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Tue Dec  8 12:55:54 2015
          State : clean, degraded
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : da0c61be:48b2e856:4c17d85b:cd614625
         Events : 0.734

    Number   Major   Minor   RaidDevice State
       0       8       39        0      active sync   /dev/sda7
       1       0        0        1      removed
       2       0        0        2      removed
       3       8       55        3      active sync   /dev/sde7
       4       0        0        4      removed


[Jocko]

:thumbup

So do this sequence:

Code: Select all

mdadm --re-add  /dev/md0 /dev/sdb7
mdadm --re-add  /dev/md0 /dev/sdc7

;post

Code: Select all

mdadm --detail /dev/md0

; stop the raid

Code: Select all

mdadm --stop /dev/md0

and retry to assemble again md0

Code: Select all

mdadm --assemble

and post again

Code: Select all

mdadm --detail /dev/md0

;

[imt17]

I did the 1st two instructions. Returned errors so posting those below and awaiting instructions 1st before proceeding to the next step, in case you need me to try something else 1st.

Code: Select all

root@fvdw-sta-kirkwood:/ # mdadm --re-add  /dev/md0 /dev/sdb7
mdadm: --re-add for /dev/sdb7 to /dev/md0 is not possible

root@fvdw-sta-kirkwood:/ # mdadm --re-add  /dev/md0 /dev/sdc7
mdadm: --re-add for /dev/sdc7 to /dev/md0 is not possible


[imt17]

I also did the next one since it doesn;t run a process just returns info:

Code: Select all

root@fvdw-sta-kirkwood:/ # mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Thu Jan  1 00:00:15 1970
     Raid Level : raid1
     Array Size : 16000 (15.63 MiB 16.38 MB)
  Used Dev Size : 16000 (15.63 MiB 16.38 MB)
   Raid Devices : 5
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Tue Dec  8 12:55:54 2015
          State : clean, degraded
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : da0c61be:48b2e856:4c17d85b:cd614625
         Events : 0.734

    Number   Major   Minor   RaidDevice State
       0       8       39        0      active sync   /dev/sda7
       1       0        0        1      removed
       2       0        0        2      removed
       3       8       55        3      active sync   /dev/sde7
       4       0        0        4      removed


[Jocko]

So we need to reset these two devices as it is a raid1, there is no risk

First attempt with sdb7:
- remove its superblock

Code: Select all

mdadm --zero-superblock /dev/sdb7

- add it in the raid

Code: Select all

mdadm --add /dev/md0 /dev/sdb7

Post

Code: Select all

 cat /proc/mdstat

and

Code: Select all

mdadm --detail /dev/md0

[imt17]

Code: Select all

root@fvdw-sta-kirkwood:/ # mdadm --zero-superblock /dev/sdb7
root@fvdw-sta-kirkwood:/ # mdadm --add /dev/md0 /dev/sdb7
mdadm: added /dev/sdb7
root@fvdw-sta-kirkwood:/ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md2 : active raid1 sda9[0] sdb9[2]
      875456 blocks [5/2] [U_U__]
     
md1 : active raid1 sda8[0] sdb8[2]
      843328 blocks [5/2] [U_U__]
     
md3 : active raid1 sde5[4]
      255936 blocks [5/1] [____U]
     
md4 : active raid5 sda2[0] sde2[4] sdc2[2] sdb2[1]
      3898947584 blocks super 1.0 level 5, 512k chunk, algorithm 2 [5/4] [UUU_U]
     
md0 : active raid1 sdb7[1] sda7[0] sde7[3]
      16000 blocks [5/3] [UU_U_]
     
unused devices: <none>


Code: Select all

root@fvdw-sta-kirkwood:/ # mdadm --detail /dev/md0
/dev/md0:
        Version : 0.90
  Creation Time : Thu Jan  1 00:00:15 1970
     Raid Level : raid1
     Array Size : 16000 (15.63 MiB 16.38 MB)
  Used Dev Size : 16000 (15.63 MiB 16.38 MB)
   Raid Devices : 5
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Tue Dec  8 15:24:32 2015
          State : clean, degraded
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

           UUID : da0c61be:48b2e856:4c17d85b:cd614625
         Events : 0.753

    Number   Major   Minor   RaidDevice State
       0       8       39        0      active sync   /dev/sda7
       1       8       23        1      active sync   /dev/sdb7
       2       0        0        2      removed
       3       8       55        3      active sync   /dev/sde7
       4       0        0        4      removed


[Jocko]

So now repeat with sdc7