nwsp2 classic gpt support

[minibike12]

I haven't used mine for 4 hours when i was away.
And the other people in the house don't know how to access it yet , and the pc's here won't find the drive automatically.
We don't have a media server that was caching files.
And my nwsp2 has only 1 file on it.
So, for my knowledge, no use for 4 hours.
When i came home from work it was still spinning and about 50 degrees C.

I will try the 2TB + loop partition then.
Maybe this will increase speed.
Is it possible for you to look into 2 partitions, sda8 and sda9 for example.

And any more progress with the U-Boot?

[Jocko]

Is it possible for you to look into 2 partitions, sda8 and sda9 for example.

Sorry minibike12, but it will take more than 5 minutes to do this ;)

[fvdw]

I haven't used mine for 4 hours when i was away.
And the other people in the house don't know how to access it yet , and the pc's here won't find the drive automatically.
We don't have a media server that was caching files.
And my nwsp2 has only 1 file on it.
So, for my knowledge, no use for 4 hours.
When i came home from work it was still spinning and about 50 degrees C.

mmm... well its strange, it should spin down, something must keep it busy. But I do not believe its the loop device. More likely something in your network. Disconnect the network cable if it then spin downs it is something in your network.
Or the daemon is not running.
Type on the command line

Code: Select all

 ps -ef | grep noflushd

It should give one line output showing that the daemon is running.

50 oC is rather hot. (There is a temparature guard in our firmware to switch it down when it got above 50 or 55 oC)
ps I use my nwsp2 on its side wall instead of the normal feets . That keeps temps at least 5 to 10 oC lower

I will try the 2TB + loop partition then.
Maybe this will increase speed.
Is it possible for you to look into 2 partitions, sda8 and sda9 for example.

everything is possible but we can not exceed 2,2 TB with a ms-dos partition table that is required by the boot loader.
Making then two partitions doesn't make sense.
As said I would make just sda8 occupying 2 TB, use for partition 8 this setting (cylinders).

Code: Select all

Partition 8 start 409 end 243200

(I just have written a file of 50 GB to the nwsp2 with an average speed of 20 MB/s)

And any more progress with the U-Boot?

I have a kernel that should be able to write to the flash memory, and I have an new compiled u-boot image (based on Lacie source code) that can be written to it and that would enable GPT support. Still waiting for someone to try it.

[Mijzelf]

When advanced format disks were new (and didn't exceed the 2TiB barrier yet) I read about an 'auto alignment' feature. On an Windows machine the first (and in most cases only) partition starts at the first cylinder (cylinder counting start at zero, for some unknown reason), which is in most cases on sector 61 or 63 (can't remember which). This is not a multiple of 8, so an ad disk in a Windows machine would be unaligned by default. This was solved by letting the first virtual sector start on an offset of the first physical sector.
I cannot remember how this behavior was triggered, I think a jumper. But maybe also by decoding sector 0 being written. First partition on cylinder 1 => offset.
It doesn't make sense that this would be in effect on a >2TiB disk, as it won't have an MBR partition table anyway, but who knows?
So you could do some speed measurements with different offsets. When I read well, the +1 option was already shot, so then remains the +512, +1024, +1536, ... ,+3584.

There is no need to format a loop device for each offset, the speedtest can be done on a lower level. You dd doesn't have speed feedback, but does the 'time' option work? Try a 'time dir' or something like that.

If 'time' works, you can do

Code: Select all

losetup -o 3686809600 /dev/loop1 /dev/sda
time dd if=/dev/zero of=/dev/loop1 bs=4096 count=262144
losetup -d /dev/loop1
losetup -o 3686810112 /dev/loop1 /dev/sda
time dd if=/dev/zero of=/dev/loop1 bs=4096 count=262144
losetup -d /dev/loop1
....

And further with offsets 3686810624, 3686811136, 3686811648, 3686812160, 3686812672, 3686813184. This writes 1GiB low-level to the loopdevice. If 'time' doesn't work, you can do

Code: Select all

date;  dd if=/dev/zero of=/dev/loop1 bs=4096 count=262144;  date

and do the math yourself.

BTW, this will destroy the current filesystem on the loop device.

[fvdw]

thanks for this tip Mijzelf if this doesn't bring improvement then it must the loop device driver.
Can there be any reason you can think of why the block device driver would be slower then writing to a partition /dev/sda8. Both are block devices :scratch

[minibike12]

Sorry minibike12, but it will take more than 5 minutes to do this ;)

I know, was just wondering if it was possible to do in a future firmware, for example.

mmm... well its strange, it should spin down, something must keep it busy. But I do not believe its the loop device. More likely something in your network. Disconnect the network cable if it then spin downs it is something in your network.
Or the daemon is not running.
Type on the command line

Code: Select all

 ps -ef | grep noflushd


It should give one line output showing that the daemon is running.

This is the output, i assume it is running.

Code: Select all

root@fvdwsl-base:/ # ps -ef | grep noflushd
root       724     1  0 Dec26 ?        00:00:11 /usr/sbin/noflushd -n 5 /dev/sda
root     18416 18306  0 09:51 ttyp0    00:00:00 grep noflushd
root@fvdwsl-base:/ #


50 oC is rather hot. (There is a temparature guard in our firmware to switch it down when it got above 50 or 55 oC)
ps I use my nwsp2 on its side wall instead of the normal feets . That keeps temps at least 5 to 10 oC lower

Indeed, but this is with the black cap off.
So just the base and bare hdd.
I think this is because it is always running at full speed.
The standard 1TB hdd used to be around 65oC i think, especially in summer.

everything is possible but we can not exceed 2,2 TB with a ms-dos partition table that is required by the boot loader.
Making then two partitions doesn't make sense.
As said I would make just sda8 occupying 2 TB, use for partition 8 this setting (cylinders).

Code: Select all

Partition 8 start 409 end 243200


(I just have written a file of 50 GB to the nwsp2 with an average speed of 20 MB/s)

Ah okey, i though that the problem was that the partitions couldn't be bigger than 2TB.
But if it is the partition table, it would be useless indeed.
NTFS is not an option i guess?
When formatted correctly, NTFS can hold more than 2.2TB

I have a kernel that should be able to write to the flash memory, and I have an new compiled u-boot image (based on Lacie source code) that can be written to it and that would enable GPT support. Still waiting for someone to try it.

Hmm okey.
I will look into JTAG after new years eve.
When i've found a save way to flash the nwsp2 when bricked, i would like to try it.

When advanced format disks were new (and didn't exceed the 2TiB barrier yet) I read about an 'auto alignment' feature. On an Windows machine the first (and in most cases only) partition starts at the first cylinder (cylinder counting start at zero, for some unknown reason), which is in most cases on sector 61 or 63 (can't remember which). This is not a multiple of 8, so an ad disk in a Windows machine would be unaligned by default. This was solved by letting the first virtual sector start on an offset of the first physical sector.
I cannot remember how this behavior was triggered, I think a jumper. But maybe also by decoding sector 0 being written. First partition on cylinder 1 => offset.
It doesn't make sense that this would be in effect on a >2TiB disk, as it won't have an MBR partition table anyway, but who knows?
So you could do some speed measurements with different offsets. When I read well, the +1 option was already shot, so then remains the +512, +1024, +1536, ... ,+3584.

There is no need to format a loop device for each offset, the speedtest can be done on a lower level. You dd doesn't have speed feedback, but does the 'time' option work? Try a 'time dir' or something like that.

If 'time' works, you can do

Code: Select all

losetup -o 3686809600 /dev/loop1 /dev/sda
time dd if=/dev/zero of=/dev/loop1 bs=4096 count=262144
losetup -d /dev/loop1
losetup -o 3686810112 /dev/loop1 /dev/sda
time dd if=/dev/zero of=/dev/loop1 bs=4096 count=262144
losetup -d /dev/loop1
....


And further with offsets 3686810624, 3686811136, 3686811648, 3686812160, 3686812672, 3686813184. This writes 1GiB low-level to the loopdevice. If 'time' doesn't work, you can do

Code: Select all

date;  dd if=/dev/zero of=/dev/loop1 bs=4096 count=262144;  date


and do the math yourself.

BTW, this will destroy the current filesystem on the loop device.

Thanks for the info!
I will test different offsets today.
Will let you know if there is a change in performance.

Btw, web interface is a bit buggy after using dd if=/dev/zero .......
Hurray, i now have a 16TB hdd, hehe :hammerhead

Code: Select all

Share Name   Volume    Total Size       Free Size
public       Default   16325.23 GB      2747.09 GB                  
openshare    Default   16325.23 GB      2747.09 GB                  


[Mijzelf]

thanks for this tip Mijzelf if this doesn't bring improvement then it must the loop device driver.
Can there be any reason you can think of why the block device driver would be slower then writing to a partition /dev/sda8. Both are block devices :scratch

The only reason I can think of is when it writes in chunks <4KiB. But I wouldn't know why, as the filesystem upon it uses 4KiB blocks. I cannot find anything in the losetup or the loop.ko manual about specifying blocksize.

Btw, web interface is a bit buggy after using dd if=/dev/zero .......

If you didn't unmount /dev/loop1 first, you've killed the filesystem while it was active. Don't know how the web interface should cope with that. :)

[fvdw]

Ah okey, i though that the problem was that the partitions couldn't be bigger than 2TB.
But if it is the partition table, it would be useless indeed.
NTFS is not an option i guess?
When formatted correctly, NTFS can hold more than 2.2TB

The problem is that the number of sectors exceed the max possible one when going beyond 2.2 TB. That why a partition cannot be bigger then 2,2 TB with ms-dos partition tables. But also two of 1.5 TB is a no go.

NTFS won't solve this on a disk with a ms-dos partition table. Furthermore the ntfs-3g driver used to write to ntfs formatted disks in the firmware is not very fast for writing. Its the free version and the designers made it slow to force you to buy the commercial version. But they wont give the source for arm platform.

[fvdw]

thanks for this tip Mijzelf if this doesn't bring improvement then it must the loop device driver.
Can there be any reason you can think of why the block device driver would be slower then writing to a partition /dev/sda8. Both are block devices :scratch

The only reason I can think of is when it writes in chunks <4KiB. But I wouldn't know why, as the filesystem upon it uses 4KiB blocks. I cannot find anything in the losetup or the loop.ko manual about specifying blocksize.

Btw, web interface is a bit buggy after using dd if=/dev/zero .......

If you didn't unmount /dev/loop1 first, you've killed the filesystem while it was active. Don't know how the web interface should cope with that. :)

mmm. then it remains a mystery why write speed is affected.

about webinterface, using dd can not not have any effect on it unless you are overwriting the system partition from which the firmware runs.... :snicker

[Jocko]

Sorry minibike12, but it will take more than 5 minutes to do this ;)

I know, was just wondering if it was possible to do in a future firmware, for example.

I began to list what changes required so that the firmware handles correctly this second data partition.
First, set this second partition is easy and no big changes needed. The created shares on this partition are available like the others via samba :-D

But a lot of changes is needed :dry :
- to restore the ftp, nfs, webdav and webexplorer features
- to restore the scan of some DLNA servers on this partition : minidlna, firefly and mediathomb
- to allow to select a file or a path in several menus : user accounts, backup, wget and HTTrack.
- .... :scratch

So I think all changes won't be done in once. ;)