Slow SMB/rsync Transfer

[oldandcrusty]

I am trying to backup a share I have on a Synology DS1812+ to my Lacie 2big v2. I am using smb to mount the synology share on the Lacie box. I am running rsync on Lacie. My transfer rate is 15-17 MBs. The synology has a specified read rate of 200MBs and the lacie has a write rate in excess of 100MBs. If I try in the other direction, mounting the Lacie share and running rsync on synology, the transfer rate falls to below 1MBs. Same if I try the z option on rsync. top on lacie shows 0-3% idle with the rest being used by 'user' and 'system'. 0% wait though. It appears my 2big is running at full bore. Is there something here I am missing? Are there any adjustments to be made? Is this the best I can hope for? :sob

My drives are SATA3, my network is GB and full duplex, firmware is fvdw 18.2, samba connection is smb2, the smb.conf is the default system generated version, samba is V4, the raid is linear. There is no iperf to test the network connection. The default smb.conf seems to well thought out by the developers. I've run out of ideas. :dontknow

mike

[fvdw]

Do you are writing files on the lacie. If it are many small files than 15-17 MB/s is not too bad. If it are large files (100 MB or higher) it could be better. But maybe rsync is the limiting factor here. Certainly when verify is in use.
What is transfer speed if you copy your share from synology nas to 2big2 using samba ? (explorer).

Ps a gigabit network can reach max 100 MB/s but because of limited cpu power and ram size, the max writing speed on 2big2 for large files will not be higher than 30 MB/s. For many small files it can drop to much lower values due to extra disk i/o to write file headers

[oldandcrusty]

I decided to try NFS. I set it up on both boxes. After a short think, I decided to make rsync run on the synology box since it has a bigger processor (Atom dual core vs. arm) so I mounted the LaCie on the synology and started up rsync. Once I hit the large file files, my transfer doubled to 30-35 MBs. Most of the files (by size, not number) are videos with the rest being thumbnails and such. I figure it's the best it will do so I am done at this point doing any more troubleshooting. :beer:

Thanks for your insight.

mike

[thekarmola]

I have the same problem using Rsync Transfer. It is for me now 5-6 mb/s transfer to the Lacie, very slow.

[Jocko]

Hi

Check if you use the option

Code: Select all

-z, --compress (compress data)

which decreases transfer speed instead of increasing it with a poor cpu !

So you should not use it on the Lacie nas

[oldandcrusty]

I almost never use compression on rsync since most of what I backup is already compressed like videos and music. I set up rsyncd on the LaCie box. Today's transfer was not a real test since most of the transfer was updating ownership and permissions. Just a few files were transfer. I read that using the rsync daemon was more efficient because rsync talks to rsync rather than having to go through a shell on the destination.

mike

[Jocko]

Note if you use rsyncd over ssh protocol then data are encrypted and transfer speed will decrease also

[uelpenich]

The rsync log file in folder fvdw of the rsyncd system shows to you information about the number of copied / modified / deletet / moved files and the amount of transferred data.

For a better understanding of the bottlenecks I suggest to read my description of the different rsync modes
https://plugout.net/viewtopic.php?f=7&t=1470.

I figured out the best case transfer rates between a Windows10 laptop (I7 processor, SSD, to a 2big2 with fvdw-sl firmware version 18-2 and Samba Version (4.9.4), 2 hard disks RAID1, wired LAN with Dell GBit Switch). I copied 16 binary files 1.048.576 KB generated by a test program h2testw_1.4 originally created by c't (Heise Verlag) to check memory sticks:
write speed: Windows10 to 2big2: 28 .. 29 MByte/s
read speed: 2big2 to Windows10: 104 MByte/s

PS:
* 104 MByte/s is quiet close to the GBit Network bottleneck
* The ARM processor has 6 fast "copy units" to copy and process (xor e.t.c., but no data encryption) large amounts of data in a simple way without cpu interaction.