big volumes

[dockeradz]

Since this is the 5 disk capable hardware that's still usable it's pretty easy to come up with a volume size that exceeds the 32bit ext4 limit of about 16tb. Like for instance if you plug in 5 8tb disks. So I thought well I can just make 2 raid 5 volumes across the 5 disks and I'll be good. 30tb / 2 is about 15tb. So I did that in ssh terminal and created the ext4 fs on each but it seems like the gui (at first I blamed the gui I'm sad to say) only sees md0. Then I started digging into the gui and discovered the api and discoveryd.

My question is how deep of a rabbit hole is this? Have additional md volumes been tried and is it possible to get the api to see them? Should I give up, bite the bullet and buy a drobo?

Speaking of drobo they somehow (and I have no idea how) have sudo 64bit ext4 support on their 32bit 5n.

[dockeradz]

Made some progress. The issue with not seeing the md0 was a bug in dm_get_raid_status, problem was unexpected line in /proc/mdstat:

Code: Select all

root@nas:/ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md1 : active raid5 sda9[0] sde9[5] sdd9[3] sdc9[2] sdb9[1]
      15622250496 blocks super 1.0 level 5, 512k chunk, algorithm 2 [5/4] [UUUU_]
         resync=DELAYED
      bitmap: 0/30 pages [0KB], 65536KB chunk

md0 : active raid5 sda8[0] sde8[5] sdd8[3] sdc8[2] sdb8[1]
      15622244352 blocks super 1.0 level 5, 512k chunk, algorithm 2 [5/4] [UUUU_]
      [========>............]  recovery = 42.5% (1660536540/3905561088) finish=1732.4min speed=21597K/sec
      bitmap: 2/30 pages [8KB], 65536KB chunk


Here is the patch:

Code: Select all

Index: dm_get_raid_status.finc
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- dm_get_raid_status.finc   (revision 877c880a9e94a2e2b5eca10d953eece9d3ca94fd)
+++ dm_get_raid_status.finc   (date 1589330533792)
@@ -50,8 +50,9 @@
             return exit(1);
          }
          $count = count($result);
-         for ($i = 1; $i < $count; $i=$i+3)
+         for ($i = 1; $i < $count; $i++)
          {
+             if (strpos($result[$i],':') == false) continue;
             $temp = explode(":",$result[$i]);
             $temp[0]=trim($temp[0]);
             $mdDevice = "/dev/".$temp[0];


The UI needs a little tweaking, but not too bad:

[fvdw]

:thumbup nice. Thanks for your feedback and suggestions. Nope we did not thought of using multiple md's to enable use disks bigger than 4 TB. Now with bigger disks becoming available at affordable prices it might be time to incorporate that option.

I will discuss with Jocko to include this option in the firmware.

Ps be aware that the 5big2 has a relative smal RAM and not so powerfull arm5 cpu so using to serve many user might not give the performance you are looking for. Also copying 32 TB of data to the 5big2 will take some time.....

[Jocko]

Thank you for the tip
Need to deepen if there is no other issue somewhere

[dockeradz]

Thanks for the replys! I'm am using this just for media for myself, mostly digitizing my blu-ray collection.

When I reboot the array the nas_conf_db.xml file gets overwritten. Then I loose the config for the additional mds. What is writing that file out and can it be stopped?

[Jocko]

Nope it is not simple, the firmware is currently built to use only one data partition by disk and many changes must be done

[dockeradz]

Here is a patch to utilize multiple data partitions on boot. (edited to put partition in the right order) I also changed the partition color to match the md vol color, to indicate what partition is a member. I haven't looked too much into the UI yet to somehow display the mds in the same table when they share disks. I made an assumption that the data partitions on local disks always start at 8. The original code takes the last partition. It looks like a big change but basically I just added a loop for any partition over 8, and moved the md vols above to get the color.

Code: Select all

Index: dm_update_single_db.finc
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- dm_update_single_db.finc   (revision 877c880a9e94a2e2b5eca10d953eece9d3ca94fd)
+++ dm_update_single_db.finc   (date 1590939296261)
@@ -106,7 +106,31 @@
       $disks=array();
       $vols=array();
       natsort($result);
-      
+
+        //add RAID volume
+        exec("/usr/bin/grep ' md' /proc/partitions|/usr/bin/awk '{print \$4}'",$raid);
+        $hasRaidVol=false;
+        foreach ($raid as $md){
+            $raidinfo=null;
+            if(dm_get_raid_settings($md,$raidinfo) == OK_E){
+                $jvol['volid']="110".$md[2];
+                $jvol['md']= '/dev/'.$md;
+                $jvol['fs']= $jvol['volid'];
+                $jvol['lastindex']= 1;
+                $volname=exec("sudo /usr/sbin/e2label /dev/$md 2>/dev/null");
+                $volname=str_replace(' ','-',$volname);
+                $jvol['name']= (empty($volname))?$raidinfo['name']:$volname;
+                $jvol['devices']= $raidinfo['devices'];
+                $jvol['raidtype']= $raidinfo['raidtype'];
+                $jvol['color']=dm_random_color();
+                $jvol['formatting']= ( $raidinfo['init'] == 'true' )?0:1;
+                $jvol['size']= dm_get_vol_size("/dev/$md",$jvol['volid']);
+                //push the new volume
+                $vols[]=$jvol;
+                $hasRaidVol=true;
+            }
+        }
+
       foreach ($result as  $key => $disk){
          unset($serial);
          unset($model);
@@ -171,61 +195,66 @@
             $jdisk['init']='false';
             $jdisk['partitions']['partition']="";
          }
-         else{
-            $nbpart=count($jpartitions);
-            if($nbpart ==0){
-               //partitions table is empty
-               $jdisk['freesize']=$jdisk['size'];
-               $jdisk['init']='false';
-               $jdisk['partitions']['partition']="";
-            }
-            elseif($jpartitions[$nbpart-1]['type']!='83'&& $jpartitions[$nbpart-1]['type']!='8300'
-                  && $jpartitions[$nbpart-1]['type']!='fd'&& $jpartitions[$nbpart-1]['type']!='FD00'){
-               //filesystem of sdx(last) is not ext3 => not valid
-               $jdisk['freesize']=$jdisk['size'];
-               $jdisk['init']='false';
-               $jdisk['partitions']['partition']="";
-            }
-            else{
-               //we can use sdx(last) as data partition
-               $size=dm_get_partition_size($jpartitions[$nbpart-1]['name'])/1024/1024;
-               $jdisk['size']=$size;
-               $jdisk['freesize']=0;
-               $jdisk['init']='true';
-               //set partition data
-               $jpart=array();
-               $jpart['lvname']='/dev/'.$jpartitions[$nbpart-1]['name'];
-               $jpart['color']=dm_random_color();
-               //check if the device belongs to a RAID array
-               unset($md);
-               exec('sudo  /sbin/mdadm --examine --brief '.$jpart['lvname'].'|/usr/bin/grep ARRAY|/usr/bin/awk \'{print $1"\n"$2}\'',$md);
-               if (isset($md[0]) && $md[0] == 'ARRAY'){
-                  $jpart['volid']='110'.trim($md[1],'/mddev');
-               }
-               else{
-                  $jpart['volid']=$fs[$ind_disk];
-                  $command="sudo /usr/sbin/blkid  -o value -s TYPE {$jpart['lvname']} 2>/dev/null";
-                  $fsdev=trim(exec($command));
-                  //set volume data
-                  $jvol['volid']=$jpart['volid'];
-                  $jvol['md']= $jpart['lvname'];
-                  $jvol['fs']= $jpart['volid'];
-                  $jvol['lastindex']= 1;
-                  $volname=exec('sudo /usr/sbin/e2label '.$jpart['lvname'].' 2>/dev/null');
-                  $volname=str_replace(' ','-',$volname);
-                  $jvol['name']= (empty($volname))?'Vol-'.strtoupper($ind_disk):$volname;
-                  $jvol['devices']= '';
-                  $jvol['raidtype']= 'single';
-                  $jvol['color']=$jpart['color'];
-                  $jvol['formatting']= ($fsdev == 'ext3' || $fsdev == 'ext4')?0:1;
-                  $jvol['size']= $size;
+         else {
+                $nbpart = count($jpartitions);
+                if ($nbpart == 0) {
+                    //partitions table is empty
+                    $jdisk['freesize'] = $jdisk['size'];
+                    $jdisk['init'] = 'false';
+                    $jdisk['partitions']['partition'] = "";
+                } elseif ($jpartitions[$nbpart - 1]['type'] != '83' && $jpartitions[$nbpart - 1]['type'] != '8300'
+                    && $jpartitions[$nbpart - 1]['type'] != 'fd' && $jpartitions[$nbpart - 1]['type'] != 'FD00') {
+                    //filesystem of sdx(last) is not ext3 => not valid
+                    $jdisk['freesize'] = $jdisk['size'];
+                    $jdisk['init'] = 'false';
+                    $jdisk['partitions']['partition'] = "";
+                } else {
+                    //we can use sdx(last) as data partition - a little presumptuous
+                    //handle multiple data partitions
+                    //data partitions on the 5big2 start at 8...is that true for other platforms?
+                    for ($nbparti = 8-1; $nbparti < $nbpart; $nbparti++) {
+                        $size = dm_get_partition_size($jpartitions[$nbparti]['name']) / 1024 / 1024;
+                        $jdisk['size'] = $size;
+                        $jdisk['freesize'] = 0;
+                        $jdisk['init'] = 'true';
+                        //set partition data
+                        $jpart = array();
+                        $jpart['lvname'] = '/dev/' . $jpartitions[$nbparti]['name'];
+                        $jpart['color'] = dm_random_color();
+                        //check if the device belongs to a RAID array
+                        unset($md);
+                        exec('sudo  /sbin/mdadm --examine --brief ' . $jpart['lvname'] . '|/usr/bin/grep ARRAY|/usr/bin/awk \'{print $1"\n"$2}\'', $md);
+                        if (isset($md[0]) && $md[0] == 'ARRAY') {
+                            $jpart['volid'] = '110' . trim($md[1], '/mddev');
+                            //get color of md
+                            foreach ($vols as $vol) {
+                                if ($vol['volid'] == $jpart['volid']) $jpart['color'] = $vol['color'];
+                            }
+                        } else {
+                            $jpart['volid'] = $fs[$ind_disk];
+                            $command = "sudo /usr/sbin/blkid  -o value -s TYPE {$jpart['lvname']} 2>/dev/null";
+                            $fsdev = trim(exec($command));
+                            //set volume data
+                            $jvol['volid'] = $jpart['volid'];
+                            $jvol['md'] = $jpart['lvname'];
+                            $jvol['fs'] = $jpart['volid'];
+                            $jvol['lastindex'] = 1;
+                            $volname = exec('sudo /usr/sbin/e2label ' . $jpart['lvname'] . ' 2>/dev/null');
+                            $volname = str_replace(' ', '-', $volname);
+                            $jvol['name'] = (empty($volname)) ? 'Vol-' . strtoupper($ind_disk) : $volname;
+                            $jvol['devices'] = '';
+                            $jvol['raidtype'] = 'single';
+                            $jvol['color'] = $jpart['color'];
+                            $jvol['formatting'] = ($fsdev == 'ext3' || $fsdev == 'ext4') ? 0 : 1;
+                            $jvol['size'] = $size;
 
-                  //push the new volume
-                  $vols[]=$jvol;
-               }
-               $jpart['index']=0;
-               $jpart['size']=$size;
-               $jdisk['partitions']['partition'][]=$jpart;
+                            //push the new volume
+                            $vols[] = $jvol;
+                        }
+                        $jpart['index'] = 0;
+                        $jpart['size'] = $size;
+                        $jdisk['partitions']['partition'][] = $jpart;
+                    }
             }
          }
          $disks[]=$jdisk;
@@ -252,30 +281,6 @@
             $vols[]=$usbvl;
       }
       
-      //add RAID volume
-      exec("/usr/bin/grep ' md' /proc/partitions|/usr/bin/awk '{print \$4}'",$raid);
-      $hasRaidVol=false;
-      foreach ($raid as $md){
-         $raidinfo=null;
-         if(dm_get_raid_settings($md,$raidinfo) == OK_E){
-            $jvol['volid']="110".$md[2];
-            $jvol['md']= '/dev/'.$md;
-            $jvol['fs']= $jvol['volid'];
-            $jvol['lastindex']= 1;
-            $volname=exec("sudo /usr/sbin/e2label /dev/$md 2>/dev/null");
-            $volname=str_replace(' ','-',$volname);
-            $jvol['name']= (empty($volname))?$raidinfo['name']:$volname;
-            $jvol['devices']= $raidinfo['devices'];
-            $jvol['raidtype']= $raidinfo['raidtype'];
-            $jvol['color']=dm_random_color();
-            $jvol['formatting']= ( $raidinfo['init'] == 'true' )?0:1;
-            $jvol['size']= dm_get_vol_size("/dev/$md",$jvol['volid']);
-            //push the new volume
-            $vols[]=$jvol;
-            $hasRaidVol=true;
-         }
-      }
-      
       //update nas database with the current volid
       $oldshares=array();
       $oldusers=array();


[Jocko]

Warning: this patch is not enough for managing the single volumes (set same volid on all data partitions on the same disk). That means you can not create the 2d raid from the web interface and you have to create it manually.