Problems with raid5

[fvdw]

after you have rebooted and loaded all the modules do the command

Code: Select all

dmesg | grep mapper

it should give something like this

Code: Select all

root@fvdwsl-5big2:/ # dmesg | grep mapper
[   16.069710] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
[   16.078354] device-mapper: multipath: version 1.6.0 loaded
[   16.083856] device-mapper: multipath round-robin: version 1.0.0 loaded
[   16.090349] device-mapper: multipath queue-length: version 0.1.0 loaded
[   16.096959] device-mapper: multipath service-time: version 0.2.0 loaded
[   16.103623] device-mapper: dm-log-userspace: version 1.3.0 loaded
[   16.109686] device-mapper: raid: Loading target version 1.5.2


[samrise]

after reboot:

Code: Select all

root@nas:/ # dmesg | grep mapper
root@nas:/ #


[fvdw]

and this command also doesn't show it ?

Code: Select all

cat /proc/devices

[samrise]

Code: Select all

root@nas:/ # cat /proc/devices
Character devices:
  1 mem
  2 pty
  3 ttyp
  4 /dev/vc/0
  4 tty
  4 ttyS
  5 /dev/tty
  5 /dev/console
  5 /dev/ptmx
  7 vcs
 10 misc
 13 input
 81 video4linux
 89 i2c
128 ptm
136 pts
180 usb
189 usb_device
252 ietctl
253 bsg
254 rtc

Block devices:
  1 ramdisk
259 blkext
  7 loop
  8 sd
  9 md
 65 sd
 66 sd
 67 sd
 68 sd
 69 sd
 70 sd
 71 sd
128 sd
129 sd
130 sd
131 sd
132 sd
133 sd
134 sd
135 sd
254 mdp
root@nas:/ #


[fvdw]

so it is not loaded, try if you can load the module

Code: Select all

modprobe dm-mod

[samrise]

Code: Select all

root@nas:/dev # modprobe dm-mod
root@nas:/dev # lsmod
dm_mod 61288 0 - Live 0xbf263000
raid10 38932 0 - Live 0xbf255000
raid456 57610 1 - Live 0xbf241000
async_raid6_recov 4754 1 raid456, Live 0xbf23c000
async_memcpy 1168 2 raid456,async_raid6_recov, Live 0xbf238000
async_pq 3686 2 raid456,async_raid6_recov, Live 0xbf234000
async_xor 2557 3 raid456,async_raid6_recov,async_pq, Live 0xbf230000
xor 3821 1 async_xor, Live 0xbf22c000
async_tx 1948 5 raid456,async_raid6_recov,async_memcpy,async_pq,async_xor, Live 0xbf228000
raid6_pq 83250 2 async_raid6_recov,async_pq, Live 0xbf210000
raid1 25990 0 - Live 0xbf205000
raid0 6859 0 - Live 0xbf200000
md_mod 97860 4 raid10,raid456,raid1,raid0, Live 0xbf1df000
uvcvideo 60902 0 - Live 0xbf1ca000
videobuf2_vmalloc 2469 1 uvcvideo, Live 0xbf1c6000
videobuf2_memops 1256 1 videobuf2_vmalloc, Live 0xbf1c2000
videobuf2_core 22544 1 uvcvideo, Live 0xbf1b8000
v4l2_common 8288 0 - Live 0xbf1b1000
v4l2_int_device 1906 0 - Live 0xbf1ad000
videodev 85579 3 uvcvideo,videobuf2_core,v4l2_common, Live 0xbf18f000
iscsi_tcp 7345 0 - Live 0xbf189000
libiscsi_tcp 11130 1 iscsi_tcp, Live 0xbf182000
libiscsi 29755 2 iscsi_tcp,libiscsi_tcp, Live 0xbf174000
scsi_transport_iscsi 32871 2 iscsi_tcp,libiscsi, Live 0xbf163000
iscsi_trgt 96019 0 - Live 0xbf143000 (O)
nfsd 81372 1 - Live 0xbf126000
exportfs 2825 1 nfsd, Live 0xbf122000
nfs 104468 0 - Live 0xbf0f9000
lockd 57065 2 nfsd,nfs, Live 0xbf0e3000
sunrpc 156449 3 nfsd,nfs,lockd, Live 0xbf0a9000
usblp 9297 0 - Live 0xbf0a2000
fuse 60769 0 - Live 0xbf08b000
ntfs 202409 0 - Live 0xbf050000
isofs 21978 0 - Live 0xbf046000
cifs 234559 0 - Live 0xbf000000
root@nas:/dev #


What next?

[fvdw]

check if the devicer mapper is loaded

Code: Select all

cat /proc/devices

If yes try to format again

[samrise]

No go.

Code: Select all

root@nas:/ # lsmod
dm_mod 61288 0 - Live 0xbf263000
raid10 38932 0 - Live 0xbf255000
raid456 57610 1 - Live 0xbf241000
async_raid6_recov 4754 1 raid456, Live 0xbf23c000
async_memcpy 1168 2 raid456,async_raid6_recov, Live 0xbf238000
async_pq 3686 2 raid456,async_raid6_recov, Live 0xbf234000
async_xor 2557 3 raid456,async_raid6_recov,async_pq, Live 0xbf230000
xor 3821 1 async_xor, Live 0xbf22c000
async_tx 1948 5 raid456,async_raid6_recov,async_memcpy,async_pq,async_xor, Live 0xbf228000
raid6_pq 83250 2 async_raid6_recov,async_pq, Live 0xbf210000
raid1 25990 0 - Live 0xbf205000
raid0 6859 0 - Live 0xbf200000
md_mod 97860 4 raid10,raid456,raid1,raid0, Live 0xbf1df000
uvcvideo 60902 0 - Live 0xbf1ca000
videobuf2_vmalloc 2469 1 uvcvideo, Live 0xbf1c6000
videobuf2_memops 1256 1 videobuf2_vmalloc, Live 0xbf1c2000
videobuf2_core 22544 1 uvcvideo, Live 0xbf1b8000
v4l2_common 8288 0 - Live 0xbf1b1000
v4l2_int_device 1906 0 - Live 0xbf1ad000
videodev 85579 3 uvcvideo,videobuf2_core,v4l2_common, Live 0xbf18f000
iscsi_tcp 7345 0 - Live 0xbf189000
libiscsi_tcp 11130 1 iscsi_tcp, Live 0xbf182000
libiscsi 29755 2 iscsi_tcp,libiscsi_tcp, Live 0xbf174000
scsi_transport_iscsi 32871 2 iscsi_tcp,libiscsi, Live 0xbf163000
iscsi_trgt 96019 0 - Live 0xbf143000 (O)
nfsd 81372 1 - Live 0xbf126000
exportfs 2825 1 nfsd, Live 0xbf122000
nfs 104468 0 - Live 0xbf0f9000
lockd 57065 2 nfsd,nfs, Live 0xbf0e3000
sunrpc 156449 3 nfsd,nfs,lockd, Live 0xbf0a9000
usblp 9297 0 - Live 0xbf0a2000
fuse 60769 0 - Live 0xbf08b000
ntfs 202409 0 - Live 0xbf050000
isofs 21978 0 - Live 0xbf046000
cifs 234559 0 - Live 0xbf000000
root@nas:/ # mdadm --stop /dev/md0
mdadm: stopped /dev/md0
root@nas:/ # mdadm --zero-superblock /dev/sda8
root@nas:/ # mdadm --zero-superblock /dev/sdb8
root@nas:/ # mdadm --zero-superblock /dev/sdc8
root@nas:/ # mdadm --zero-superblock /dev/sde8
root@nas:/ # mdadm --zero-superblock /dev/sdd8
root@nas:/ #  mdadm --create --verbose --assume-clean --force /dev/md0 --level=raid5 --raid-devices=5 /dev/sda8 /dev/sdb8 /dev/sdc8 /dev/sdd8 /dev/sde8
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 64K
mdadm: /dev/sdb8 appears to contain an ext2fs file system
    size=973474740K  mtime=Thu Jan  1 01:00:00 1970
mdadm: /dev/sdc8 appears to contain an ext2fs file system
    size=973474740K  mtime=Thu Jan  1 01:00:00 1970
mdadm: /dev/sdd8 appears to contain an ext2fs file system
    size=973474740K  mtime=Thu Jan  1 01:00:00 1970
mdadm: /dev/sde8 appears to contain an ext2fs file system
    size=973474740K  mtime=Thu Jan  1 01:00:00 1970
mdadm: size set to 973474624K
Continue creating array? y
mdadm: array /dev/md0 started.
root@nas:/ # cat /etc/mdadm.0 > /etc/mdadm.conf
root@nas:/ # mdadm --detail --scan
ARRAY /dev/md0 level=raid5 num-devices=5 UUID=f75e87c9:bebee079:3494dc27:48e0eef8
root@nas:/ # mdadm --detail --scan >> /etc/mdadm.conf
root@nas:/ # cat /proc/mdstat
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid5 sde8[4] sdd8[3] sdc8[2] sdb8[1] sda8[0]
      3893898496 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]

unused devices: <none>
root@nas:/ # mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Mon Aug 17 01:19:33 2015
     Raid Level : raid5
     Array Size : 3893898496 (3713.51 GiB 3987.35 GB)
    Device Size : 973474624 (928.38 GiB 996.84 GB)
   Raid Devices : 5
  Total Devices : 5
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Mon Aug 17 01:19:33 2015
          State : clean
 Active Devices : 5
Working Devices : 5
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : f75e87c9:bebee079:3494dc27:48e0eef8
         Events : 0.1

    Number   Major   Minor   RaidDevice State
       0       8       72        0      active sync   /dev/sda8
       1       8       56        1      active sync   /dev/sdb8
       2       8       40        2      active sync   /dev/sdc8
       3       8       24        3      active sync   /dev/sdd8
       4       8        8        4      active sync   /dev/sde8
root@nas:/ # mke2fs -t ext3 -j -m 1 -b 4096 /dev/md0
mke2fs 1.42.11 (09-Jul-2014)
Creating filesystem with 973474624 4k blocks and 243376128 inodes
Filesystem UUID: 921ddaa0-1426-4f48-9a3a-adac5e904a0a
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done
Writing inode tables:  1755/29709

Around this point on dmesg:

Code: Select all

device-mapper: ioctl: 4.24.0-ioctl (2013-01-15) initialised: dm-devel@redhat.com
md: md0: resync done.
md: checkpointing resync of md0.
md0: detected capacity change from 3987352059904 to 0
md: md0 stopped.
md: unbind<sda8>
md: export_rdev(sda8)
md: unbind<sde8>
md: export_rdev(sde8)
md: unbind<sdd8>
md: export_rdev(sdd8)
md: unbind<sdc8>
md: export_rdev(sdc8)
md: unbind<sdb8>
md: export_rdev(sdb8)
md: bind<sda8>
md: bind<sdb8>
md: bind<sdc8>
md: bind<sdd8>
md: bind<sde8>
md/raid:md0: device sde8 operational as raid disk 4
md/raid:md0: device sdd8 operational as raid disk 3
md/raid:md0: device sdc8 operational as raid disk 2
md/raid:md0: device sdb8 operational as raid disk 1
md/raid:md0: device sda8 operational as raid disk 0
md/raid:md0: allocated 5282kB
md/raid:md0: raid level 5 active with 5 out of 5 devices, algorithm 2
RAID conf printout:
 --- level:5 rd:5 wd:5
 disk 0, o:1, dev:sda8
 disk 1, o:1, dev:sdb8
 disk 2, o:1, dev:sdc8
 disk 3, o:1, dev:sdd8
 disk 4, o:1, dev:sde8
md0: detected capacity change from 0 to 3987352059904
 md0: unknown partition table
attempt to access beyond end of device
sdb8: rw=0, want=3283362188, limit=1946949480
attempt to access beyond end of device
sdc8: rw=0, want=3283362188, limit=1946949480
attempt to access beyond end of device
sdb8: rw=4096, want=3283362188, limit=1946949480
attempt to access beyond end of device
sde8: rw=0, want=3283362188, limit=1946949480
attempt to access beyond end of device
sdd8: rw=0, want=3283362188, limit=1946949480
attempt to access beyond end of device
sdc8: rw=4096, want=3283362188, limit=1946949480
attempt to access beyond end of device
sda8: rw=0, want=3283362188, limit=1946965545
md/raid:md0: Disk failure on sdc8, disabling device.
md/raid:md0: Operation continuing on 4 devices.
md/raid:md0: Disk failure on sdb8, disabling device.
md/raid:md0: Operation continuing on 3 devices.
Buffer I/O error on device md0, logical block 38273305
lost page write due to I/O error on md0
RAID conf printout:
 --- level:5 rd:5 wd:3
 disk 0, o:1, dev:sda8
 disk 1, o:0, dev:sdb8
 disk 2, o:0, dev:sdc8
 disk 3, o:1, dev:sdd8
 disk 4, o:1, dev:sde8
RAID conf printout:
 --- level:5 rd:5 wd:3
 disk 0, o:1, dev:sda8
 disk 1, o:0, dev:sdb8
 disk 3, o:1, dev:sdd8
 disk 4, o:1, dev:sde8
RAID conf printout:
 --- level:5 rd:5 wd:3
 disk 0, o:1, dev:sda8
 disk 1, o:0, dev:sdb8
 disk 3, o:1, dev:sdd8
 disk 4, o:1, dev:sde8
RAID conf printout:
 --- level:5 rd:5 wd:3
 disk 0, o:1, dev:sda8
 disk 3, o:1, dev:sdd8
 disk 4, o:1, dev:sde8
Buffer I/O error on device md0, logical block 38273040
lost page write due to I/O error on md0
Buffer I/O error on device md0, logical block 38273056
lost page write due to I/O error on md0
Buffer I/O error on device md0, logical block 38273072
lost page write due to I/O error on md0
Buffer I/O error on device md0, logical block 38273041
lost page write due to I/O error on md0
Buffer I/O error on device md0, logical block 38273057
lost page write due to I/O error on md0
Buffer I/O error on device md0, logical block 38273073
lost page write due to I/O error on md0
Buffer I/O error on device md0, logical block 38240768
lost page write due to I/O error on md0
Buffer I/O error on device md0, logical block 38240769
lost page write due to I/O error on md0


[fvdw]

seems we are still missing a few modules that must be loaded

Code: Select all

modprobe dm-multipath
modprobe dm-raid

[fvdw]

possible all of these must be loaded

device-mapper modules.JPG

sorry for this trouble, but on 5big1 the kernel that can be loaded at boot must be kept smaller then 2 MB therefor all these extra features must be loaded as modules afterwards.
As I do not have a 5big1 myself and have little experience with raid I also need traial and error to see what must be loaded to get raid working properly
On the kirkwood devices everything is already present in the kernel but those devices can load bigger kernels at boot