[SOLVED] After a firmware upgrade, blinking blue, lost 5Big2

[fvdw]

well be glad that the reset didn't work with the red light, that would have wiped all the data on your disks.
Anyhow I can give you an instruction to try to get it alive again, however no guarantees are given. It should leave your data untouched as long as you keep your fingers away from md4 containing the raid arry build from all sdx2 partitions. If we don't succeed we still can mount the array and copy the dat to somewhere else, although with 14TB that is not so easy..
Are you prepared to take the risk ?

[Glaven Clattuck]

Are you prepared to take the risk ?

yessss i'm ready!

[fvdw]

as first step lets agree on the definition of the raid array's
md0 = sd[a]..[e]7
md1 = sd[a]..[e]8
md2 = sd[a]..[e]9
md3 = sd[a]..[e]5
md4 = sd[a]..[e]2 =< your data

Be sure to use these names when assembling the arrays because I will use them in the instruction, in case you have assigned other names , well then it is obvious what will happen.

lets first see what is on md0, md1 and md2 before overwriting them with an older version
Assemble the arrays and mount them and do a directory listing (ls -al) of the mountpoints.

[Glaven Clattuck]

as first step lets agree on the definition of the raid array's
md0 = sd[a]..[e]7
md1 = sd[a]..[e]8
md2 = sd[a]..[e]9
md3 = sd[a]..[e]5
md4 = sd[a]..[e]2 =< your data

Be sure to use these names when assembling the arrays because I will use them in the instruction, in case you have assigned other names , well then it is obvious what will happen.

lets first see what is on md0, md1 and md2 before overwriting them with an older version
Assemble the arrays and mount them and do a directory listing (ls -al) of the mountpoints.

I think that already made this. read my post #9

[fvdw]

:disapprove no you didn't after mounting the arrays, you did it before mounting the arrays

so please..

[Jocko]

@fvdw

about raid array all is ok and built as you asked

Code: Select all

root@fvdw-sta-kirkwood:/ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md3 : active raid1 sda5[0] sde5[4] sdb5[3] sdd5[2] sdc5[1]
      255936 blocks [5/5] [UUUUU]
     
md2 : active raid1 sda9[0] sde9[4] sdb9[3] sdd9[2] sdc9[1]
      875456 blocks [5/5] [UUUUU]
     
md1 : active raid1 sda8[0] sde8[4] sdb8[3] sdd8[2] sdc8[1]
      843328 blocks [5/5] [UUUUU]
     
md0 : active raid1 sda7[0] sdd7[4] sde7[3] sdc7[2] sdb7[1]
      16000 blocks [5/5] [UUUUU]
     
md4 : active raid5 sda2[0] sde2[4] sdb2[2] sdd2[3] sdc2[1]
      15619969024 blocks super 1.0 level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
     
unused devices: <none>

@Glaven Clattuck

But you need to do this

Code: Select all

mount

and

Code: Select all

ls -al /md0
ls -al /md1
ls -al /md2

[Glaven Clattuck]

:disapprove no you didn't after mounting the arrays, you did it before mounting the arrays

so please..

this is

Code: Select all

root@fvdw-sta-kirkwood:/ # mdadm --assemble /dev/md0 /dev/sd[abcde]7
mdadm: /dev/sda7 is busy - skipping
mdadm: /dev/sdb7 is busy - skipping
mdadm: /dev/sdc7 is busy - skipping
mdadm: /dev/sdd7 is busy - skipping
mdadm: /dev/sde7 is busy - skipping
root@fvdw-sta-kirkwood:/ # mdadm --assemble /dev/md1 /dev/sd[abcde]8
mdadm: /dev/sda8 is busy - skipping
mdadm: /dev/sdb8 is busy - skipping
mdadm: /dev/sdc8 is busy - skipping
mdadm: /dev/sdd8 is busy - skipping
mdadm: /dev/sde8 is busy - skipping
root@fvdw-sta-kirkwood:/ # mdadm --assemble /dev/md2 /dev/sd[abcde]9
mdadm: /dev/sda9 is busy - skipping
mdadm: /dev/sdb9 is busy - skipping
mdadm: /dev/sdc9 is busy - skipping
mdadm: /dev/sdd9 is busy - skipping
mdadm: /dev/sde9 is busy - skipping
root@fvdw-sta-kirkwood:/ # mdadm --assemble /dev/md3 /dev/sd[abcde]5
mdadm: /dev/sda5 is busy - skipping
mdadm: /dev/sdb5 is busy - skipping
mdadm: /dev/sdc5 is busy - skipping
mdadm: /dev/sdd5 is busy - skipping
mdadm: /dev/sde5 is busy - skipping
root@fvdw-sta-kirkwood:/ # mdadm --assemble /dev/md4 /dev/sd[abcde]2
mdadm: /dev/sda2 is busy - skipping
mdadm: /dev/sdb2 is busy - skipping
mdadm: /dev/sdc2 is busy - skipping
mdadm: /dev/sdd2 is busy - skipping
mdadm: /dev/sde2 is busy - skipping


[fvdw]

as Jocko mentioned, provide the output of

Code: Select all

ls -al /md0
ls -al /md1
ls -al /md2

[Glaven Clattuck]

@fvdw

about raid array all is ok and built as you asked

Code: Select all

root@fvdw-sta-kirkwood:/ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md3 : active raid1 sda5[0] sde5[4] sdb5[3] sdd5[2] sdc5[1]
      255936 blocks [5/5] [UUUUU]
     
md2 : active raid1 sda9[0] sde9[4] sdb9[3] sdd9[2] sdc9[1]
      875456 blocks [5/5] [UUUUU]
     
md1 : active raid1 sda8[0] sde8[4] sdb8[3] sdd8[2] sdc8[1]
      843328 blocks [5/5] [UUUUU]
     
md0 : active raid1 sda7[0] sdd7[4] sde7[3] sdc7[2] sdb7[1]
      16000 blocks [5/5] [UUUUU]
     
md4 : active raid5 sda2[0] sde2[4] sdb2[2] sdd2[3] sdc2[1]
      15619969024 blocks super 1.0 level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
     
unused devices: <none>

@Glaven Clattuck

But you need to do this

Code: Select all

mount

and

Code: Select all

ls -al /md0
ls -al /md1
ls -al /md2

there we are

Code: Select all

root@fvdw-sta-kirkwood:/ # mount
rootfs on / type rootfs (rw,size=254924k,nr_inodes=63731)
none on /proc type proc (rw,relatime)
none on /sys type sysfs (rw,relatime)
none on /dev/pts type devpts (rw,relatime,mode=600)
/dev/md1 on /md1 type ext3 (rw,relatime,errors=continue,barrier=1,data=ordered)
/dev/md0 on /md0 type ext3 (rw,relatime,errors=continue,barrier=1,data=ordered)
/dev/md4 on /md4 type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /md2 type ext3 (rw,relatime,errors=continue,barrier=1,data=ordered)


and

Code: Select all

root@fvdw-sta-kirkwood:/ # ls -al /md1
total 104
drwxr-xr-x   22 root     root          4096 Oct 28  2014 .
drwxrwxrwt   16 root     root           440 Jun  6 17:51 ..
drwxr-xr-x    2 root     root          4096 Oct 28  2014 bin
drwxr-xr-x    2 root     root          4096 Oct 28  2014 boot
drwxrwxr-x    2 root     root          4096 Oct 28  2014 dev
drwxr-xr-x   28 root     root          4096 Oct 28  2014 etc
drwxr-xr-x    2 root     100           4096 Dec 12  2011 home
drwxr-xr-x    3 root     root          4096 Oct 28  2014 lacie
drwxr-xr-x    7 root     root          4096 Oct 28  2014 lib
drwx------    2 root     root         16384 Oct 28  2014 lost+found
drwxr-xr-x    2 root     root          4096 Dec 12  2011 media
drwxr-xr-x    2 root     root          4096 Dec 12  2011 mnt
drwxr-xr-x    2 root     root          4096 Dec 12  2011 opt
drwxr-xr-x    2 root     root          4096 Dec 12  2011 proc
drwxr-x---    2 root     root          4096 Oct 28  2014 root
drwxr-xr-x    2 root     root          4096 Oct 28  2014 sbin
drwxr-xr-x    2 root     root          4096 Dec 12  2011 share
drwxr-xr-x    2 root     root          4096 Dec 12  2011 sys
drwxrwxrwt    2 root     root          4096 Dec 12  2011 tmp
drwxr-xr-x   13 root     root          4096 Oct 28  2014 usr
drwxr-xr-x   14 root     root          4096 Oct 28  2014 var
drwxr-xr-x   11 root     root         12288 Oct 28  2014 www


second

Code: Select all

root@fvdw-sta-kirkwood:/ # ls -al /md2
total 28
drwxr-xr-x    5 root     root          4096 May 13 20:14 .
drwxrwxrwt   16 root     root           440 Jun  6 17:51 ..
drwxr-xr-x    4 root     root          4096 May 22 15:49 EDMINI
drwx------    2 root     root         16384 May 13 20:14 lost+found
drwxr-xr-x    3 root     root          4096 May 13 20:14 snaps


and last

Code: Select all

root@fvdw-sta-kirkwood:/ # ls -al /md0
total 30
drwxr-xr-x   20 root     root          1024 Oct 28  2014 .
drwxrwxrwt   16 root     root           440 Jun  6 17:51 ..
drwxr-xr-x    2 root     root          1024 Oct 28  2014 bin
drwxr-xr-x    2 root     root          1024 Dec 12  2011 boot
drwxr-xr-x    2 root     root          1024 Oct 28  2014 dev
drwxr-xr-x    4 root     root          1024 Oct 28  2014 etc
drwxr-xr-x    3 root     root          1024 Oct 28  2014 home
drwxr-xr-x    2 root     root          1024 Dec 12  2011 include
drwxr-xr-x    4 root     root          1024 Oct 28  2014 lib
drwx------    2 root     root         12288 Oct 28  2014 lost+found
drwxr-xr-x    2 root     root          1024 Dec 12  2011 mnt
drwxr-xr-x    2 root     root          1024 Dec 12  2011 opt
drwxr-xr-x    2 root     root          1024 Dec 12  2011 proc
drwxr-xr-x    2 root     root          1024 Dec 12  2011 root
drwxr-xr-x    2 root     root          1024 Jun  2 18:46 sbin
drwxr-xr-x    2 root     root          1024 Dec 12  2011 snapshots
drwxr-xr-x    2 root     root          1024 Dec 12  2011 sys
drwxr-xr-x    2 root     root          1024 Dec 12  2011 tmp
drwxr-xr-x    4 root     root          1024 Oct 28  2014 usr
drwxr-xr-x    6 root     root          1024 Oct 28  2014 var


not in order but is wath you ask :-)

[Glaven Clattuck]

--- cut for duplication ---