[School of Physics - Optics Group]
 

CAUTION!

The rsync provided here is very VERY old. There are many known bugs, and some of these may cause data corruption. Dan Barrett has alerted me to one such bug; see here. Please use one of the alternative Windows-based rsync solutions, for example:

Installing ssh and rsync on a Windows machine: minimalist approach

Note: again, this stuff is very old. To install OpenSSH on Windows, see: This page provides a brief outline of how to get rsync running on a Windows system. This is for the simplest self-contained system without a full Cygwin installation. If you want Cygwin as well, please install Cygwin with openssh and rsync: cygwin.

My version of rsync+ssh is a very stock-standard rsync, with the cygwin1.dll (currently version 1.3.6-6), and a simplified version of ssh. It is version 1.2.26 (yeah, ok, old, but works well enough for this). The main difference is that you can put the whole thing in one directory, wherever you like (e.g. c:\rsync or c:\program files\rsync). And you specify your username and home directory with environment variables (e.g. SET USERNAME=bloggs), i.e. you don't need /etc/passwd. You also don't need /usr, /usr/bin etc.

Downloads

Binaries are available here. Please do not use the rsync binaries unless you are willing to corrupt your destination files. If you are willing to accept that risk, you will find .ZIP and .tar.gz versions of a basic package (rsync, ssh, cygwin1.dll) (734kB) or a full package (1.4MB), and individual files. Note that cygwin1.dll is a completely unaltered version 1.3.6-6.

Source files are here, in .tar.bz2. Note that the cygwin source is a completely unaltered version 1.3.6-6, and the rsync sources are also completely clean. The ssh package is modified a bit to use environment variables instead of an /etc/passwd file.

Installation (Windows 95/98/NT4/2k/XP)

To use rsync, you first have to have ssh running. Don't even think about using rsh, and you probably don't want to rely on the built-in rsync transport.
  1. Unpack rsync_minimal or rsync_full, e.g. as c:\rsync.

  2. Edit the rsetup.bat file. This is a simple batch file that defines your username and home directory and path; modify the appropriate two lines.

  3. Open a DOS or NT command window and go to the appropriate directoy, e.g. cd \rsync

  4. Set the path etc. by running rsetup.


Running rsync

  1. Try rsync:
    where remote is the name of a remote host (e.g. a Linux box, or optics).
    This will save the c:\rsync directory and files to machine remote, at backupdir under your home directory on remote.


  2. For a full drive, e.g. D: try:
    This will save all of D:\* to remote:backupdir/mypc/d/*.

    Note 1: rsync will only create directories one deep. So in the example above, it would create remote:backupdir/mypc/d if remote:backupdir/mypc already exists, but not if only remote:backupdir exists (i.e. without mypc). Note 2: These examples use a directory relative to your home directory on remote. You can also use absolute directories, e.g. remote:/rsync/data2/blogs.

    WARNING!

    Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!


  3. Try the example batch file to back up a whole drive, drivec.bat. Edit this to set the appropriate remote hostname and remote directory location. Note the directory notation, e.g. /cygdrive/c for drive C:\.

    You can just execute this batch files to mirror your drive. For example, make a shortcut to it from an icon on your desktop, and then just double-click on the shortcut and your whole drive will be mirrored exactly.


  4. Other things to try:

WARNING!

Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!

Setting up ssh keys

Normally you will have to enter your password every time. However, you can generate privacy keys for ssh such that you won't have to enter your password. This is not secure, and if someone gets the identity key on yorur PC, they can log on to the remote host as you, and wreak havoc. There is a mechanism for protecting against this, but for the moment I don't have time to explain. Hence, use this at your own risk.
  1. Generate your security keys by running ssh-keygen:
    You will be asked Enter file in which to save the key (/rsync/.ssh/identity):, just hit Enter.

    WARNING: This creates a ssh key without a passphrase. That means that anyone can use your Win95/98 machine to connect to any of the hosts that you set up, without having to type a password ever (because Win95/98 don't have real passwords). If anyone else has access to your PC, then use ssh-keygen and enter a passphrase when prompted.


  2. Add \rsync\.ssh\identity.pub to the end of your ~/.ssh/authorized_keys file on the system you want to send your files to. Let's assume this is a machine called remote.

    Copy identity.pub to remote with a command like:
    where you need to replace myusername with your username.

    Then log in to remote, and:

    WARNING: Once this is done, anyone that has a copy of your private key, \rsync\.ssh\identity (not your public key, identity.pub) can log in to remote without a password. So, make sure you don't let anyone read this file. In particular, make sure that if you "share" your drive, you have it set so that a password is needed for both read and write.

  3. In the DOS or CMD window, run ssh:
    You will be prompted to ask if you want to add remote to the authorized hosts, answer is "yes" (all three letters). You should then be connected to remote. You can use it like a telnet session (e.g. try ls, and so on).


Recovering files

You can recover files by logging on to remote, and copying the files, or by finding remote in your Network Neighbourhood, or using rsync itself. For example: will list the files in remote:/rsync/data1/d. You can recover one with something like: where the . can be replaced with any local destination.

Automating rsync

This is most easily done with the Windows Task Scheduler. If you have an old Windows, you can get Task Scheduler by downloading the free Internet Explorer (version 4 or later). To add a scheduled task:
  1. click on My Computer and look at the bottom. You should see a folder named Scheduled Tasks
  2. click on Scheduled Tasks
  3. click on Add Scheduled Task
  4. follow the menus; when offered a list of programs to run, select Browse.. and find a batch file such as c:\rsync\drivec.bat
  5. select appropriate times and so on

Running multiple rsyncs simultaneously

Steve Brown discovered that you can only run one rsync at a time, unless you use the -P option for ssh.

Excluding directories

Including and excluding files and directories is well described in the rsync man page. For some good help on this stuff, look at this message by David Bolen.

WARNING! If doing this on a PC, be careful that your --exclude-from or --include-from file is in Unix format (newlines rather that cr/lf). You have been warned! Put a dos2ux line in your rsync script to ensure this. Again, you have been warned!

WARNING! Make sure source directory ends with a / (in this case, /users/), and make sure destination directory does not (in this case /rsync/users).

It can all be very confusing, and two things are not terribly clear. First of all, the basic idea. To exclude various files and directories, you can create a file listing them, and execute rsync with the appropriate option, for example: Let's assume the exclude.txt file looks like: To understand how this works, you should first of all read the man page, and remember:
  1. rsync would compare the source filename "usera" to "+ /usera/". That is, it doesn't compare "/users/usera". The head of the file tree is "/users".
  2. rsync doesn't compare all filenames with the list. That is, it doesn't recursively go into subdirectories once it thinks a directory has been excluded. So in the example above, the "+ /userf" tells rsync to match that subdirectory but not the files and subdirectories in it.
  3. Note that the "- /userb/" and "- /userd/" patterns are redundant; these would be excluded anyway by the final "- /*".

Some "gotchas" in working with rsync under Windows

  1. Include/Exclude file lists
    On windows, your --exclude-from or --include-from file must be in Unix format (newlines rather that cr/lf). Put a dos2ux line in your rsync script to ensure this.


  2. Timestamps
    Windows doesn't keep file modification times to better than two seconds. Use the --modify-window=2 option to get around this.


  3. Time zones
    rsync tries to convert all file timestamps into UTC at both ends. If seemingly identical files with the same timestamp seem to be transferring, try this at both ends:
    If the remote and local files have the same timestamp, and they're still transferring, then you have a different problem. Otherwise, try --modify-window=3601 which means ignore time differences of less than an hour. This can be very helpful after a daylight savings time change.


  4. cygwin using ascii mode to read files
    Sometimes cygwin decides to open files in ascii format (i.e. convert newlines to cr/lf pairs). If this occurs, you need to mount your local drive (the one with the files you want to rsync) in binmode. This seems to be a bit unreliable but as far as I can tell, this is supposed to work. Try one of the following (the -s option adds the mount to the cygwin section of the registry, so it should "stick").

Merging filesets (e.g. laptop+desktop)

This is still experimental. The instructions given here are inefficient in terms of disk use, but safe.

Before you start, it's a good idea to look at the how the files are stored on both machines. For example, if on your laptop you have d:\personal\photos and your desktop has d:\mystuff\photos, then the merged fileset will have both, even if "photos" are the same in both cases. Also, rsync is case sensitive, so if a file is Mydir on one drive and mydir on the other, your merged set will include both. A little effort at homologising your directory structures beforehand can save you lots of time and disk space.

Let's assume you want to merge two filesets, laptop drive D and desktop drive D.
  1. First, make mirrors of the two computers you want to merge, to a server, e.g. a computer called remote:
  2. Now make a copy of one of these on remote. For example, copy the laptop to a new area which we'll call merge:
  3. Now merge that with the other (i.e. desktop) using rsync: Note the -u option. This says update only so that if a file already exists and is newer, don't overwrite it. Also, the -b option means that old files are backed up (a ~ is added to the backup filename).
  4. Just to be really paranoid, you can do the reverse and see if both merged sets are the same. While still logged on to remote:
  5. At this point you have two hopefully identical merged filesets. You should carefully go through and see if they're what you think they should be. Do search for those files that were backed up (e.g. find /rsync/data1/joeblog/merge -name "*~"). If you're happy with what has happened, you can do it "in real life", for example using rsync to do the merge on your desktop or on your laptop, using host remote as the source. Here's how you might want to do it (I haven't tried it yet!). From your laptop: and from your desktop:
Created: 5 January 2000
Updated: 21 April 2001
Maintainer: Robert Scholten, School of Physics, Email:  r.scholten at physics.unimelb.edu.au
Authorised by: Robert Scholten, School of Physics, Email:  r.scholten at physics.unimelb.edu.au
This page, its contents and style, are the responsibility of the author and do not represent the views, policies or opinions of The University of Melbourne.