Booting a GPT disk with a non-GPT aware u-boot

[Mijzelf]

I managed to boot fwdw-sl firmware from a GPT disk, using an unpatched NS2 u-boot.

First create a GPT disk, using parted:

Code: Select all

# parted /dev/sdc unit s print

Model: Packard Bell Carbon (scsi)
Disk /dev/sdc: 160836480s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start     End         Size        File system  Name  Flags
 1      2048s     1003519s    1001472s    ext3         83
 2      1003520s  2007039s    1003520s    ext3         83
 3      2007040s  3012607s    1005568s                 83
 4      3012608s  3013671s    1064s                    5
 5      3014656s  4583423s    1568768s    ext3         83
 6      4585472s  4601855s    16384s                   83
 7      4603904s  5607423s    1003520s    ext3         83
 8      5609472s  160835583s  155226112s  ext3         83

Fill it with fvdw-sl, as usual. The kernel needs to be GPT aware, I used UIMAGE-26394-NWSP2CL-51.

Parted also created a protective MBR:

Code: Select all

# fdisk -l /dev/sdc

Disk /dev/sdc: 82.3 GB, 82348277760 bytes
255 heads, 63 sectors/track, 10011 cylinders, total 160836480 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1   160836479    80418239+  ee  GPT

Use sfdisk to resize the first partition, start 0, size 1. Then use fdisk to create partitions 4, 5 and 6:

Code: Select all

# fdisk -l /dev/sdc


Disk /dev/sdc: 82.3 GB, 82348277760 bytes
255 heads, 63 sectors/track, 10011 cylinders, total 160836480 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks  Id System
/dev/sdc1               1       16064        8032  ee GPT
/dev/sdc4         3012608   160836479    78911936   5 Extended
/dev/sdc5         3014656     4583423      784384  83 Linux
/dev/sdc6         4585472     4601855        8192  83 Linux

The start of partition 4, and the start and end of partition 5 and 6 are carefully copied from the GPT listing.

Note: sfdisk also generated a garbage partition 2, which I deleted using fdisk. Don't know what triggered this behavior.

Now insert the disk in the NS2 (Or whatever, I think this will work for any fvdw-sl supported disk.) This is done using a 80GB disk, but I see no reason why it wouldn't work with a >2TiB disk, as long as partition 6 is below the 2TiB barrier.

Background:
An GPT disk also has an protective MBR partition with type ee, which starts at sector 1, and spans the whole disk (or 2TiB, whatever is smaller). When a disk has a partition 1 type ee starting at sector 1, an GPT aware OS will look at sector 2 for a GPT, and ignore the MBR. A non GPT aware OS (or bootloader) will just interpret the MBR. The u-boot in this box will just search for parition 6, and ignore everything else.

/dev/sdc4 is an extended partition, but GPT doesn't support extended partitions. And extended partition is actually just an extra partition table, located at the given (start) sector. /dev/sdc4 is not used by fvdw-sl, so I could just put that partition table there. The extended partition is needed, partition 6 can only be a logical partition, when using an MBR.

Note: This works with a Linux kernel. I don't know if other GPT aware OSses will all be happy with this construction. (Which is called a Hybrid MBR, BTW)

[fvdw]

:applause nice work Mijzelf, you already told this idea some time ago and indeed it seems to work. I looked at the "hybrid" solution but cracking u-boot was one of the challenges I wanted to accomplish. :whistle (+ to add the possibility that it first checks usb for boot to make recovery easier or prepare a new disk inside the nwsp2 without the need for a PC running linux.

There might be a problem with a disk > 2.2 TB and that is the last_usable against last LBA check of U-boot (see the topic where we developed the patched u-boot). If a GPT disk is bigger then 2.2Tb then u-boot found that the last usable LBA number was bigger then the last LBA and declared the partition table invalid and stops (red/blue led coming on). It gets this info from mv-ide driver in u-boot and passes this to the routine checking if the partition table is valid or not. In case of ms-dos table this is a different routine then GPT so maybe the check is not done. In you hybrid MBR the end sector of partition 4 might spoil the party. (I noticed that you took as end sector for partition 4 the end sector of the disk, for a disk bigger then 2.2 TB this will not be possible of course. But I guess you could choose end sector of partition 4 just a little bigger then the end sector of partition 6 to prevent this problem.

I had a quick look in the part_dos.c file and it doesn't seem to do the same checks as gpt so probably no issue.

But in the end only a trial with a disk bigger then 2.2TB can tell if it works or not

ps now knowing that flashing u-boot is possible and that we have a working u-boot with gpt support makes the decision to take the risk to flash u-boot less difficult. But I agree flashing always has a risk, for instance if you get a power failure during flashing or when the file goes corrupt in the transfer process However I flashed my nwsp2 the last week about 60 times without any problem.

[Mijzelf]

Yes, I think you are right about the end sector of partition 4. It can be placed shortly after partition 6.

I did some experiments with a loop device:

Code: Select all

root@fvdwsl-base:/ # losetup -o 8589934592 /dev/loop1 /dev/sda
root@fvdwsl-base:/ # time dd if=/dev/zero of=/dev/loop1 bs=4096 count=64k
65536+0 records in
65536+0 records out
real   0m 20.79s
user   0m 0.06s
sys   0m 1.76s
root@fvdwsl-base:/ # time dd if=/dev/zero of=/dev/sda seek=2097152 bs=4096 count=64k   
65536+0 records in
65536+0 records out
real   0m 4.28s
user   0m 0.06s
sys   0m 1.54s
root@fvdwsl-base:/ # time dd if=/dev/zero of=/dev/sda seek=16777216 bs=512 count=512k   
524288+0 records in
524288+0 records out
real   2m 26.54s
user   0m 0.48s
sys   0m 8.76s
root@fvdwsl-base:/ # time dd if=/dev/zero of=/dev/loop1 bs=512 count=512k 
524288+0 records in
524288+0 records out
real   0m 36.38s
user   0m 0.47s
sys   0m 6.56s

This all writes 256MiB at offset 8GiB of the disk. What is striking is that the difference in speed between loop device and direct access is much bigger for 4k blocks than for 512b blocks. Further that for 4k blocks the user and sys time for both loop and direct access is almost equal, while the real time is totally different.

My first thought on the bad performance of a loop device on an AF disk was that the loopdevice would write 512b blocks, no matter what the input block size is. But that is not the case, I think. In that case the user&sys time for the loop@4k would be bigger.the user&sys time for direct access@4k. Somewhere the 4k block should have to be cut.
And also @512b the loopdevice is significanty slower, while it even uses less user&sys time.

Any thoughts?

[fvdw]

no I have no idea other then it must be a difference in the driver despite that both are a "block" device. But that there is a difference you can see for instance using Windows task manager, you can use it to see network performance when writing or reading a file. In case of writing to partition is rather stable at a level around 20 MB/s (larger file). However when using a loop device you see the speed go up and down. It seems the loop device writes then waits for something and then continues and so on. Why it waits I don't know. Attached a screenshot. The first graph is writing via a loop device (actually a disk image on disk mounted using a loop device) the second is writing to normal ext3 partition

loop-part.JPG