Pushing Files from Windows to Linux/Unix Hosts with cwRsync
“Use the best tool for the job” is a great principle. I however reserve the right to define which tool is the best when the person doing the job is going to be me. That is why I develop my Web properties, such as this blog, on a Windows PC, as I am more comfortable with Windows as a desktop platform, but for a very similar reason I run them on a Linux VPS.
This in particular means I need to deploy from Windows to Linux. Back then, I manually copied the new and changed files using the WinSCP plugin for FAR, and that was okay while there were just a few files. I also have a staging environment — a VirtualBox VM that more or less replicates my VPS setup, so I could have set up some shared folders on that VM, and then use
rsync to push changes from staging to production. But instead I have set up
rsync to push files right from Windows to either staging or production. Here is how you can do that too:
Setting Up cwRsync
- Download and install the last free version of cwRsync.
If you are on a 64-bit system, open the
cwrsync.cmdtemplate script found in the root of your cwRsync installation in a text editor and change
Setting Up the Remote Host
To secure your
rsync transfers, you’ll need to run them over SSH, but I presume that you don’t want to type a password each time. With public key authentication, you may create a separate passphrase-less key and restrict its use to
rsync as follows:
On your Windows system, create a new SSH key pair without a passphrase.
ssh-keygen(there is a copy in the
bin\directory of your cwRsync installation):
ssh-keygen.exe -N "" -f key-name -C "comment"
This will create files
key-namein the current directory (see the note below). These files contain public and private SSH keys respectively, the latter also known as identification. For instance:
D:\> %homedrive% C:\> cd %homepath% C:\Users\Me\> if not exist .ssh mkdir .ssh C:\Users\Me\> cd .ssh C:\Users\Me\.ssh> "C:\Program Files (x86)\cwRsync\bin\ssh-keygen.exe" -N "" -f rsync-only -C "rsync-only" Generating public/private rsa key pair. Your identification has been saved in rsync-only. Your public key has been saved in rsync-only.pub. The key fingerprint is: . . .Note: The default location for OpenSSH files such as keys is the
%HOME%\.sshsubdirectory of the user’s home directory. On Linux/Unix,
HOMEenvironment variable normally points to that directory.
cwrsync.cmdtemplate script therefore defines
which translates to
C:\Users\my Windows usernameon my Windows 7 box.
But if you have the
HOMEenvironment variable already set, most likely by CygWin or some other Windows-Unix compatibility/interoperability package, you may wish to comment out that line from
Log in to your Linux box as the user that will be pushing files. That must be an unprivileged account with write permissions for the target directory(ies.)
If that user cannot log in remotely, log in using an administrative account and use
sudoto run an interactive shell as that user:
sudo -s -H -u user
If the user account in question does not yet have public key authentication configured, create a directory called
.sshin its home directory, copy the public key into
~/.ssh/authorized_keysand restrict permissions:
chmod -R go= ~/.ssh
Otherwise, append the public key to the existing
As there is no passphrase on the private key, anyone who manages to take hold of it can now do an SSH login to your site. To restrict what can be done using that key, use the
rrsyncPerl script. It is normally installed in
/usr/local/binand symlinked to
/usr/bin. On my servers, it was already present in
/usr/share/doc/rsync/scripts/in gzipped form, so I installed it as follows:
sudo cp /usr/share/doc/rsync/scripts/rrsync.gz /usr/local/bin sudo gzip -d /usr/local/bin/rrsync.gz sudo chmod a+x /usr/local/bin/rrsync sudo ln -s /usr/local/bin/rrsync /usr/bin
You may also download rrsync from the official rsync site.
Now edit the
authorized_keysfile and insert the following before the “rsync-only” key:
command="/usr/local/bin/rrsync subdir" ssh-rsa ...
subdiris the subdirectory that will appear as root to
rsyncclients connecting using this key. For instance:
command="/usr/local/bin/rrsync /srv/www" ssh-rsa ... rsync-only
You may further restrict access to a few known hosts and/or subnets as follows:
from="patterns",command="/usr/local/bin/rrsync subdir" ssh-rsa ...
patternsis a comma-separated list of host names and IP addresses that are permitted to authenticate using this key, with “?” and “*” serving as wildcards:
from="192.168.0.*,myotherhost.com",command="/usr/local/bin/rrsync /srv/www" ssh-rsa ... rsync-only
Note: The more obscure way of restricting access is via a
Match Userblock in
Customizing the cwRsync script
Back on the Windows system, copy the
cwrsync.cmd template script to the desired location and append your custom
rsync command to the copy:
rsync options source destination
rsync has dozens of interdependent
options (refer to the official man page on samba.org for details), but the two essential ones are:
-e "ssh -i private-key"or
-rsh "ssh -i private-key"
forces all communications to be conducted via the specified remote shell.
private-keyis the full path to the private key in CygWin notation:
-e "ssh -i /cygdrive/c/Users/me/.ssh/rsync-only"
Windows file/directory permissions do not quite map onto Unix, so it is necessary to explicitly specify the desired permissions. For example:
grants read/write permissions to file/directory owner and group, explicitly removing the execute bit from files and setting it for directories.
Most likely, you will also want to use the following options:
- to copy subdirectories recursively;
- to compress files and conserve bandwidth;
to delete files no longer present at
to exclude files matching
pattern– may be specified more than once, e.g.:
--exclude ".git" --exclude "cwrsync*"
to test your
rsynccommand line without actually copying anything.
source is the path to the source directory, again in CygWin notation:
Note: The trailing slash can make a difference! If
source ends with a directory name without a trailing slash, that directory itself will be copied to
destination. If there is a trailing slash, the contents of that directory will be copied. If however
source ends with the current (
. ) or parent (
.. ) directory reference, the contents of the respective directoy are are synced to
destination regardless of the presence of a trailiing slash.
destination has format:
user is the name of the user on the target Linux/Unix
path is the path relative to the directory in
rrsync argument (if any), in Unix notation:
Tip: Use the caret character (
^) to break long lines:
rsync.exe -riz ^ -e "ssh -i /cygdrive/c/Users/me/.ssh/rsync-only" ^ --delete ^ --delete-excluded ^ --exclude ".git" ^ --exclude "cwrsync.*" ^ --chmod=Dug=rwX,Fug=rw,Fug-x ^ --log-file=cwrsync.log ^ . email@example.com:wp-content/themes/MyTheme/
That’s it. Let me know in the comments if it worked for you.