Error creating Raid 5

[Jocko]

Hi b1gnas

I wait what I asked here : viewtopic.php?f=30&t=2266&start=150#p20053 because you need to get first a clean raid5 with your 4disks for the next steps.

Note: About the dmesg errors, it seems that you have a controller issue with the port 5:0. can you post

Code: Select all

cat /sys/block/sda/device/queue_depth

[b1gnas]

Code: Select all

root@fvdwsl-base:/ # mdadm --examine /dev/sdb8
/dev/sdb8:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : fa58a2b1:a56fc1c1:2a5b9325:5626151a
  Creation Time : Wed Nov 16 13:22:21 2011
     Raid Level : raid5
    Device Size : 973474624 (928.38 GiB 996.84 GB)
     Array Size : 2920423872 (2785.13 GiB 2990.51 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 0

    Update Time : Thu Sep 24 13:00:48 2015
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 1
  Spare Devices : 0
       Checksum : a843ebf0 - correct
         Events : 0.107

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     0       8       56        0      active sync   /dev/sdb8

   0     0       8       56        0      active sync   /dev/sdb8
   1     1       0        0        1      faulty removed
   2     2       8       24        2      active sync   /dev/sdd8
   3     3       8        8        3      active sync   /dev/sde8
root@fvdwsl-base:/ # mdadm --examine /dev/sdc8
/dev/sdc8:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : fa58a2b1:a56fc1c1:2a5b9325:5626151a
  Creation Time : Wed Nov 16 13:22:21 2011
     Raid Level : raid5
    Device Size : 973474624 (928.38 GiB 996.84 GB)
     Array Size : 2920423872 (2785.13 GiB 2990.51 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0

    Update Time : Wed Nov 16 13:00:27 2011
          State : active
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0
       Checksum : a103b937 - correct
         Events : 0.2

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     1       8       40        1      active sync   /dev/sdc8

   0     0       8       56        0      active sync   /dev/sdb8
   1     1       8       40        1      active sync   /dev/sdc8
   2     2       8       24        2      active sync   /dev/sdd8
   3     3       8        8        3      active sync   /dev/sde8
root@fvdwsl-base:/ # mdadm --examine /dev/sdd8
/dev/sdd8:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : fa58a2b1:a56fc1c1:2a5b9325:5626151a
  Creation Time : Wed Nov 16 13:22:21 2011
     Raid Level : raid5
    Device Size : 973474624 (928.38 GiB 996.84 GB)
     Array Size : 2920423872 (2785.13 GiB 2990.51 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 0

    Update Time : Thu Sep 24 13:00:48 2015
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 1
  Spare Devices : 0
       Checksum : a843ebd4 - correct
         Events : 0.107

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     2       8       24        2      active sync   /dev/sdd8

   0     0       8       56        0      active sync   /dev/sdb8
   1     1       0        0        1      faulty removed
   2     2       8       24        2      active sync   /dev/sdd8
   3     3       8        8        3      active sync   /dev/sde8
root@fvdwsl-base:/ # mdadm --examine /dev/sde8
/dev/sde8:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : fa58a2b1:a56fc1c1:2a5b9325:5626151a
  Creation Time : Wed Nov 16 13:22:21 2011
     Raid Level : raid5
    Device Size : 973474624 (928.38 GiB 996.84 GB)
     Array Size : 2920423872 (2785.13 GiB 2990.51 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 0

    Update Time : Thu Sep 24 13:00:48 2015
          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 1
  Spare Devices : 0
       Checksum : a843ebc6 - correct
         Events : 0.107

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     3       8        8        3      active sync   /dev/sde8

   0     0       8       56        0      active sync   /dev/sdb8
   1     1       0        0        1      faulty removed
   2     2       8       24        2      active sync   /dev/sdd8
   3     3       8        8        3      active sync   /dev/sde8
root@fvdwsl-base:/ #


i never removed any disk... :scratch

[b1gnas]

Code: Select all

root@fvdwsl-base:/ # cat /sys/block/sda/device/queue_depth
31


[Jocko]

It seems that sdc8 has issue.

1/Try to re-add sdc8
- remove it

Code: Select all

mdadm --manage /dev/md0 --remove /dev/sdc8

- clean its superblock

Code: Select all

mdadm --zero-superblock /dev/sdc8

- re-add

Code: Select all

mdadm --re-add /dev/md0 /dev/sdc8

Post the output of these commands and also

Code: Select all

cat /proc/mdstat
mdadm --detail /dev/md0

[b1gnas]

Code: Select all

root@fvdwsl-base:/ # mdadm --manage /dev/md0 --remove /dev/sdc8
mdadm: hot remove failed for /dev/sdc8: No such device or address
root@fvdwsl-base:/ #


[Jocko]

:whistle Indeed it is alreadt removed => forget this step

[b1gnas]

Code: Select all

root@fvdwsl-base:/ # mdadm --zero-superblock /dev/sdc8
root@fvdwsl-base:/ # mdadm --re-add /dev/md0 /dev/sdc8
mdadm: added /dev/sdc8
root@fvdwsl-base:/ # cat /proc/mdstat
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sdc8[4] sdb8[0] sde8[3] sdd8[2]
      2920423872 blocks level 5, 64k chunk, algorithm 2 [4/3] [U_UU]
      [>....................]  recovery =  0.0% (0/973474624) finish=1014036.0min speed=0K/sec

unused devices: <none>
root@fvdwsl-base:/ # mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Wed Nov 16 13:22:21 2011
     Raid Level : raid5
     Array Size : 2920423872 (2785.13 GiB 2990.51 GB)
    Device Size : 973474624 (928.38 GiB 996.84 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Sep 24 13:52:36 2015
          State : clean, degraded, recovering
 Active Devices : 3
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 64K

 Rebuild Status : 0% complete

           UUID : fa58a2b1:a56fc1c1:2a5b9325:5626151a
         Events : 0.109

    Number   Major   Minor   RaidDevice State
       0       8       56        0      active sync   /dev/sdb8
       4       8       40        1      spare rebuilding   /dev/sdc8
       2       8       24        2      active sync   /dev/sdd8
       3       8        8        3      active sync   /dev/sde8
root@fvdwsl-base:/ #


[b1gnas]

recovery speed now is grown

Code: Select all

root@fvdwsl-base:/ # cat /proc/mdstat
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sdc8[4] sdb8[0] sde8[3] sdd8[2]
      2920423872 blocks level 5, 64k chunk, algorithm 2 [4/3] [U_UU]
      [>....................]  recovery =  0.0% (742624/973474624) finish=985.4min speed=16451K/sec

unused devices: <none>
root@fvdwsl-base:/ #


[Jocko]

do you still have a zero speed?

[b1gnas]

no no

Code: Select all

root@fvdwsl-base:/ # cat /proc/mdstat
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sdc8[4] sdb8[0] sde8[3] sdd8[2]
      2920423872 blocks level 5, 64k chunk, algorithm 2 [4/3] [U_UU]
      [>....................]  recovery =  0.4% (4453472/973474624) finish=952.9min speed=16947K/sec

unused devices: <none>