Installing ssh and rsync on a Windows machine: minimalist approach
This page provides a brief outline of how to get rsync running on a Windows system. This is for the simplest self-contained system without a full Cygwin installation. If you want Cygwin as well, please install Cygwin with openssh and rsync: cygwin.
Binaries
rsync_minimal.zip ZIP of basic package (rsync, ssh, cygwin1.dll)
rsync_minimal.tgz tar.gz of basic package (rsync, ssh, cygwin1.dll)
binaries directory
Source
To use rsync, you first have to have ssh running. Don't even think
about using rsh, and you probably don't want to rely on the built-in
rsync transport.
- Unpack rsync_minimal or rsync_full, e.g. as c:\rsync.
-
Edit the rsetup.bat file. This is a simple batch file that defines your username and home directory and path; modify the appropriate two lines.
-
Open a DOS or NT command window and go to the appropriate directoy, e.g. cd \rsync
-
Set the path etc. by running rsetup.
-
Try rsync:
rsync -e ssh -av --delete "/rsync" remote:backupdir
where remote is the name of a remote host (e.g. a Linux box, or optics).
This will save the c:\rsync directory and files to machine remote, at backupdir under your home directory on remote.
-
For a full drive, e.g. D: try:
rsync -e ssh -av --delete "/cydrive/d/" remote:backupdir/mypc/d
This will save all of D:\* to remote:backupdir/mypc/d/*.
Note 1: rsync will only create directories one deep. So in the example above, it would create remote:backupdir/mypc/d if remote:backupdir/mypc already exists, but not if only remote:backupdir exists (i.e. without mypc).
Note 2: These examples use a directory relative to your home directory on remote. You can also use absolute directories, e.g. remote:/rsync/data2/blogs.
WARNING!
Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!
-
Try the example batch file to back up a whole drive, drivec.bat. Edit this to set the appropriate remote hostname and remote directory location.
Note the directory notation, e.g. /cygdrive/c for drive C:\.
You can just execute this batch files to mirror your drive. For
example, make a shortcut to it from an icon on your desktop, and then
just double-click on the shortcut and your whole drive will be
mirrored exactly.
-
Other things to try:
- -b (i.e. -avb) makes backup copies of any files at the destination, before updating.
- -u (i.e. -avu) updates - won't overwrite newer files.
- -S (i.e. -avS) is more efficient if you have
sparse files.
- rsync -e ssh -av --delete "/cygdrive/d/" remote:/rsync/data1/d >>rsync.log will log the transfers to file rsync.log
- -z (i.e. -avz) is very useful if doing it by modem. This compresses the information before transport. Waste of time if on a fast network.
- -P (i.e. -avP) shows progress during transfers, and also keeps partially transferred files so that if the transfer is interrupted mid-file, the bit already transferred isn't wasted. Again, good if rsyncing by modem.
- The --exclude option is very useful to exclude swap files. You may also want to exclude things like netscape caches. For example:
--exclude 'pagefile.sys' --exclude 'WIN386.SWP' --exclude 'Cache/'
- --backup-dir=PATH puts all old versions of files into the named directory, thus giving an incremental backup.
- Look at the
rsync man page for more ideas.
WARNING!
Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!
Normally you will have to enter your password every time. However,
you can generate privacy keys for ssh such that you won't
have to enter your password. This is not secure, and if someone gets
the identity key on yorur PC, they can log on to the remote
host as you, and wreak havoc. There is a mechanism for protecting
against this, but for the moment I don't have time to explain. Hence,
use this at your own risk.
-
Generate your security keys by running ssh-keygen:
You will be asked Enter file in which to save the key (/rsync/.ssh/identity):, just hit Enter.
WARNING: This creates a ssh key without a passphrase. That means that anyone can use your Win95/98 machine to connect to any of the hosts that you set up, without having to type a password ever (because Win95/98 don't have real passwords). If anyone else has access to your PC, then use ssh-keygen and enter a passphrase when prompted.
- Add \rsync\.ssh\identity.pub to the end of your
~/.ssh/authorized_keys file on the system you want to send
your files to. Let's assume this is a machine called remote.
Copy identity.pub to remote with
a command like:
copy \rsync\.ssh\identity.pub \\remote\myusername\.ssh
where you need to replace myusername with your username.
Then log in to remote, and:
cd ~
cat identity.pub >>.ssh/authorized_keys
rm identity.pub
WARNING: Once this is done, anyone that has a copy of your private key, \rsync\.ssh\identity (not your public key, identity.pub) can log in to remote without a password. So, make sure you don't let anyone read this file. In particular, make sure that if you "share" your drive, you have it set so that a password is needed for both read and write.
-
In the DOS or CMD window, run ssh:
You will be prompted to ask if you want to add remote to the authorized hosts, answer is "yes" (all three letters).
You should then be connected to remote. You can use it like a telnet session (e.g. try ls, and so on).
You can recover files by logging on to remote, and copying
the files, or by finding remote in your Network Neighbourhood, or using rsync itself. For example:
rsync -e ssh -a remote:/rsync/data1/d
will list the files in remote:/rsync/data1/d. You can recover one with something like:
rsync -e ssh -a remote:/rsync/data1/d/testfile.txt .
where the . can be replaced with any local destination.
This is most easily done with the Windows Task Scheduler.
If you have an old Windows, you can get Task Scheduler by downloading the free Internet Explorer (version 4 or later).
To add a scheduled task:
- click on My Computer and look at the bottom. You should see a folder named Scheduled Tasks
- click on Scheduled Tasks
- click on Add Scheduled Task
- follow the menus; when offered a list of programs to run, select Browse.. and find a batch file such as c:\rsync\drivec.bat
- select appropriate times and so on
Steve Brown discovered that you can only run one rsync at a time, unless you use the -P option for ssh.
Including and excluding files and directories is well described in the rsync man page. For some good help on this stuff, look at this message by David Bolen.
It can all be very confusing, and two things are not terribly clear.
First of all, the basic idea. To exclude various files and
directories, you can create a file listing them, and execute rsync
with the appropriate option, for example:
rsync -e ssh -axv --delete --exclude-from=exclude.txt /users/ remote:/rsync/users
Let's assume the exclude.txt file looks like:
+ /usera/
- /userb/
+ /userc/
- /userd/
+ /userf
+ /userf/fsubdir/
- /userf/*
- /*
To understand how this works, you should first of all read the man page, and remember:
- rsync would compare the source filename "usera" to "+ /usera/". That is, it doesn't compare "/users/usera". The head of the file tree is "/users".
- rsync doesn't compare all filenames with the list. That is, it doesn't recursively go into subdirectories once it thinks a directory has been excluded. So in the example above, the "+ /userf" tells rsync to match that subdirectory but not the files and subdirectories in it.
- Note that the "- /userb/" and "- /userd/" patterns are redundant; these would be excluded anyway by the final "- /*".
This is still experimental. The instructions given here are inefficient in terms of disk use, but safe.
Before you start, it's a good idea to look at the how the files are stored on both machines. For example, if on your laptop you have d:\personal\photos and your desktop has d:\mystuff\photos, then the merged fileset will have both, even if "photos" are the same in both cases. Also, rsync is case sensitive, so if a file is Mydir on one drive and mydir on the other, your merged set will include both. A little effort at homologising your directory structures beforehand can save you lots of time and disk space.
Let's assume you want to merge two filesets, laptop drive D and desktop drive D.
- First, make mirrors of the two computers you want to merge, to a server, e.g. a computer called remote:
rsync -e ssh -avbu --delete "\\d/" remote:/rsync/data1/joeblog/laptop/d
rsync -e ssh -avbu --delete "\\d/" remote:/rsync/data1/joeblog/desktop/d
- Now make a copy of one of these on remote. For example, copy the laptop to a new area which we'll call merge:
- ssh remote
- cd /rsync/data1/joeblog
- mkdir merge
- cp -a laptop/d merge (might take awhile!)
- Now merge that with the other (i.e. desktop) using rsync:
- rsync -auvb /rsync/data1/joeblog/desktop/d/ /rsync/data1/joeblog/merge/d
Note the -u option. This says update only so that if a file already exists and is newer, don't overwrite it. Also, the -b option means that old files are backed up (a ~ is added to the backup filename).
- Just to be really paranoid, you can do the reverse and see if both merged sets are the same. While still logged on to remote:
- cd /rsync/data1/joeblog
- mkdir merge2
- cp -a desktop/d merge2 (again, might take a long time)
- rsync -auvb /rsync/data1/joeblog/laptop/d/ /rsync/data1/joeblog/merge2/d
- At this point you have two hopefully identical merged filesets. You should carefully go through and see if they're what you think they should be. Do search for those files that were backed up (e.g. find /rsync/data1/joeblog/merge -name "*~"). If you're happy with what has happened, you can do it "in real life", for example using rsync to do the merge on your desktop or on your laptop, using host remote as the source. Here's how you might want to do it (I haven't tried it yet!). From your laptop:
- rsync -auv remote:/rsync/data1/joeblog/desktop/d/ "\\d/"
and from your desktop:
- rsync -auv remote:/rsync/data1/joeblog/laptop/d/ "\\d/"
Created: 5 January 2000
Updated: 21 April 2001
Maintainer: Robert Scholten, School of Physics, Email: r.scholten@physics.unimelb.edu.au
Authorised by: Robert Scholten, School of Physics, Email: r.scholten@physics.unimelb.edu.au
This page, its contents and style, are the responsibility of the author and do not represent the views, policies or opinions of The University of Melbourne.