Help for the backup menu

 

Note : You can't make a backup if the source or destination folders contain one or more single quote.

  

With this menu, three types of backup (remote or local backup) are possible :

·          Mirror Backup

'Destination' folder is synchronized to be identical to 'Source' folder, and the files in the destination folder not present in the source folder are removed without backup.

·         Incremental Backup

'Destination' folder is synchronized to be identical to 'Source' folder, the files previous version and the removed files are saved in a time-stamped folder)

·         Restore

Same operation as 'Mirror Backup' but you can keep the extraneous files of destination folder and ignore files with a date newer than the source file.

 

A. Optional Settings:

·         Copy only source folder content :

use this option to avoid creating an additional directory level at the destination with the source folder name. Be careful: if you use a miror backup, all extra files and folders of destination (not present in source) will be deleted!!! (use the restore backup to change this behavior by selecting the option "Keep extraneous files of destination folder")
Note: this option is disabled with an incremental backup

·         Simulate the backup :

Perform a trial run with no changes made.  It is deactivated if the share "fvdw" doesn't exist.
A log is stored in the share "fvdw" (basename of the source folder)

·          Keep extraneous files of destination folder

Available only with a backup "restore"
This forces rsync to keep  any files which exist on the destination but not in the source.

·         Ignore files with a date newer than the source file

Available only with a backup "restore"
This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file.

·         Ignore the modification dates of files (compare only file size)

By default, rsync transfers files either a changed size or a changed last-modified time, this forces rsync to just looking for files that have changed in size. 

·         Backup subfolders and files

This forces  rsync to copy folders recursively

·         Exclude files

This allows you to specify excluding patterns. Patterns must be entered in the text box, separated by ":"
They will be saved in a file "source_basename.txt" in the share "fvdw" (or in "cronjobs"  if you schedule the backup).

Note :     spaces in end and beginning of pattern  are removed except if the pattern  is framed by simple quote (').
                If pattern must contain a simple quote in beginning or end of motif, the pattern must also be framed by simple quotes (ex: `' foo*' <=> pattern : `foo*: Filename starting with `foo)

Pattern :

·         *.o           : would exclude all names matching *.o

·          /foo         : would exclude a file (or directory) named foo in the source-root folder

·         foo/          : would exclude any folder named foo

·         /foo/*/bar : would exclude any file named bar which is at two levels below a folder named foo in the source-root folder

·         /foo/** /bar :  would exclude any file named bar two or more levels below a folder named foo in the source-root source

·         ...

B.    Backup from or to a remote host

With version 15-2, you have now several ways to make backups from a remote host :

• using a SSH access
• adding a remote shares to the NAS if you have a Samba or NFS server running on the remote host
  To mount theses shares on the NAS server, click on the button "Add a remote share" and see the help file of this menu.
• adding a remote rsync server if it is running on the remote host
  To mount theses shares on the NAS server, click on the button "Add a remote share" and see the help file of this menu.

The handiest method for backups over the LAN and the fastest is to use a rsync server.

Caution : if you use a remote samba share as destination to backup the NAS data, permissions and last change date file are synchronized if the remote host is a Linux/Unix OS but not the file owner. All new file will have as owner, the username used to mount the share. If you want a full backup, you must use a method describe in the chapter E. Remote backup with a SSH access or use a rsync server without a remote-shell connection.

Important Note: It is advised not to use the remote share method  for remote backup over Internet: data transfer and login without encryption! In this configuration it is better to use backup via SSH access or a remote rsync server over a remote-shell connection.  However with the limited resources of the NAS (nwsp1 and spd8020) the transfer speed can be very slow (<2Mb/s <=> 250kB/s!). On other nas (nwsp2) the speed will be higher but encryption takes time. Rsync server over a remote-shell connection is faster than a backup using a SSH access.

C.    To execute the backup

You can:

Either run the backup immediately (click on the `Accept' button)

Either schedule the backup if you are the admin (Click on the `Cron' button)

 

The table "Backup in progress" lists the backups running. If necessary, you can kill one or more backups by clicking on the button "Abort" after selecting the appropriate backups in the table.

Note:

1. Only two backups can be run simultaneously
2. To make the backup, a script file is generated by the menu in the tmp directory or in cronjobs share. Indeed it is required for e-mail notification and to test if the remote share is available before running the backup otherwise the data is written to the NAS!
3. If you rename the php files, saved in cronjobs share, keep the prefix 'Rsync_' so that the backup can be detected and listed in the table "Backup in progress".
4. If you are a non admin user, you can kill only yours backups
5. The button `Cron' is deactivated:
• If the `cronjobs' share doesn't exist
• If the selected source or destination folder is the root folder
• If the destination folder is a subfolder of  source
• If the option "simulate" is activated (nothing to plan)
• If you are a non admin user
6. The button `Accept' is deactivated:
• If two backup or more (!) are in progress
• If the selected source or destination folder is the root folder
• If the destination folder is a subfolder of  source
• the remote rsync server is both as source and destination
7. If destination filesystem is FAT32, the backup will fail with the first file larger than 4GB

D.    E-Mail Notification

        If you have enabled the E-mail notification feature (menu 'Mailer configuration'), an e-mail will send when the backup is completed or if it failed to start.

E. Optional Commands with the destination folder :

·         create new folder

You can create a new folder in the destination folder.
However not in the root of the file system. That means you must have selected a destination folder first before you can create a new folder.

·         Assign Read/Write permissions

You can assign full permissions on the destination folder (and subfolders).
When you backup a folder from a FAT or NTFS file system, permissions aren't usually sufficient : for example to delete the backup. This command is available only with the internal shares and use it only if you want rewrite files or delete. Of course, the permissions are restored after a new backup !

F.    Remote backup with a SSH access

You can't do remote backups using this menu. However, you can run remote backups from a shell in two configurations :

a)Backup is executed by a remote host that attempts an access to the SSH server (dropbear) of the NAS.
b)Backup is executed by the NAS that attempts an SSH access to a Remote Host.

Note: You can find some usefull guides how to use rsync with advanced options on the plugout board (Rsync: Guides & Tutorials)

Case "a", step to do :

on the Remote Host :

- Install a ssh client for linux (or window)

- Install rsync (if linux) or equivalent window (cwRync)

on the NAS,

- Install the RSA public key used by the Remote Host (see Help for SSH access menu)

on  the Remote Host,

- use these command patterns (linux):

rsync -e "[yourclientSSH [options]]" -azv... source_on_the_remote_host  root@youripnas:destination_on_the_nas (copy Host => Nas)

or

rsync -e "[yourclientSSH [options]]" -azv... root@youripnas:source_on_the_nas  destination_on_the_remote_host (copy NAS => Host)

 

with :

- destination_on_the_nas (or source_on_the_nas) : full path on the nas (/share/1000/sharename/subfolder/...)

- yourclientSSH : fullname of your SSH client

- [options] : options for your SSH (usually path of your rsa key and SSH port used by dropbear)

Case "b", step to do :

on the Remote Host :

- Install a SSH server for linux (or window)

- Install rsync (if linux) or equivalent window (cwRync)

- Install the RSA public key of the NAS

on the NAS,

- either from a script or a Shell Terminal use these command patterns (linux):

rsync -e "/usr/bin/dbclient -p [yourSSH port] -i dropbear_rsa_host_key" -azv... source_on_the_nas   login@youriphost:destination_on_the_remote_host (copy NAS => Host)

or

rsync  -e "/usr/bin/dbclient -p [yourSSH port] -i dropbear_rsa_host_key" -azv... login@youriphost:source_on_the_remote_host  destination_on_the_nas (copy host => Nas)

 

with :

- destination_on_the_nas (or source_on_the_nas): full path on the nas (/share/1000/sharename/subfolder/...)

- login : name authorized to do an SSH access by your remote host

- yourSSH port : SSH port used by the SSH server of the remote host

 

Note: if the remote SSH serveur uses the same port as SSH server of NAS, option"-e" is useless in this configuration, because the environment variable "RSYNC_RSH" is set

-With SSH access with key

    RSYNC_RSH=/usr/bin/dbclient -p [SSH port] -i /rw_fs/etc/dropbear/dropbear_rsa_host_key

-With SSH access with password

RSYNC_RSH=/usr/bin/dbclient -p [SSH port]

RSYNC_RSH=/usr/bin/dbclient -p [SSH port]

 

So the command to use is simplified:

For exemple : rsync  -azv... source_on_the_nas  login@youriphost:destination_on_the_remote_host

G.    Additional information 

For advanced users, standard rsync commands used :

 Backup (mirror)

/usr/sbin/rsync --stats --force --ignore-errors --modify-window=2 -HltD[pog]v[n][r/r --filter "- */*/*"] -delete[-excluded --exclude-from=motif_exclude_file.txt] [--size-only] Source Target > file.log 2>&1

[p]: used only if the source and destination filesystems are linux systems  

[og]: used only if the source and destination filesystems are linux systems and if the destination is a NAS share. 

--modify-window=2: compatibility with the precision of the dates of systems FAT (2s) 

[n] : simulate the backup 

[r] : Backup subfolders and files. If the option is not checked, substituted by [-r --filter "- */*/*"]. 

[-excluded --exclude-from=pattern_exclude_file.txt] :
            added if the option "exclude" is checked

motif_exclude_file.txt:

/share/1000/fvdw/basename(source).txt, if fvdw exist and backup not planned

/share/1000/cronjobs/basename(source).txt, if backup will be planned

/tmp/var/log/rsync/ basename(source).txt, if fvdw doesn't exist

[--size-only] : test only the filesize

file.log:

/share/1000/fvdw/basename(source)_YYYY-MM-DD_hh-mm-ss.log, if fvdw exist and backup not planned else /share/1000/cronjobs/basename(source)_YYYY-MM-DD_hh-mm-ss.log

/tmp/var/log/rsync/basename(source)_YYYY-MM-DD_hh-mm-ss.log, if fvdw doesn't exist

 Incremental Backup

 One more parameters :

[ -b --backup-dir=basename(source)_YYYY-MM-DD_hh-mm-ss]

basename(source)_YYYY-MM-DD_hh-mm-ss : folder created if there are files deleted or previous version

Restore

One more parameters :

[u]: Not update the file if the modification date of the destination file is newer

 [--delete[-excluded ]] : parameter not used if "Keep extraneous files of destination folder" is checked