[School of Physics - Optics Group]

Installing ssh and rsync on a Windows machine

Links to sections below:  

What is RSYNC?

RSYNC is a program for copying files between networked computers. It is "special" because if the same file already exists, it copies only the differences between the files, therefore saving network traffic. Thus, you can make a mirror copy of a hard drive, and keep the two copies identical over a modem link.

For more information, look at: NOTE: rsync is licensed under the standard GNU GPL. The sources are available at rsync home page.

Installation (Windows 95/98/NT4)

Several versions of rsync are available here. If downloading these (or any other items linked from this page) you should also download the GNU General Public License. The package mentioned below has the first three; choose your favourite and copy it to rsync.exe.
Some versions of cygwin1.dll are also here. If downloading these (or any other items linked from this page) you should also download the GNU General Public License. NOTE: cygwin is licensed under the standard GNU GPL. The sources are available at sources.redhat.com.

To use rsync, you have to first have secure shells running. This is a little tedious, but once installed, you can forget about it.
  1. Copy the rsync_package.zip file (and also the GNU General Public License) and unzip the package. Make sure you extract all the files, and Use folder names. That is, put the files in their appropriate directories: \etc\passwd as c:\etc\passwd and everything else into c:\rsync. Due to problems with recent releases of rsync the default rsync.exe may not work. Copy the version you want to try (e.g. rsync2.3.2.exe) to c:\rsync\rsync.exe.


  2. Edit the \etc\passwd file.

  3. Open a DOS or NT command window and cd \rsync


  4. Set the path by running p. This is a batch file (p.bat) which adds c:\rsync to your path.


  5. Generate your security keys by running ssh-keygen:
    You will be asked Enter file in which to save the key (/rsync/.ssh/identity):, just hit Enter.

    WARNING: This creates a ssh key without a passphrase. That means that anyone can use your Win95/98 machine to connect to any of the hosts that you set up, without having to type a password ever (because Win95/98 don't have real passwords). If anyone else has access to your PC, then use ssh-keygen and enter a passphrase when prompted.


  6. Add \rsync\.ssh\identity.pub to the end of your ~/.ssh/authorized_keys file on the system you want to send your files to. Let's assume this is torte. Since your home directory is the same on torte and optics you can do it on optics instead.

    For example, first copy identity.pub to optics with a command like:
    where I'm assuming n:\ is your home directory on optics.

    Then log in to optics, and:

    WARNING: Once this is done, anyone that has a copy of your private key, \rsync\.ssh\identity (not your public key, identity.pub) can log in to optics/torte without a password. So, make sure you don't let anyone read this file. In particular, make sure that if you "share" your drive, you have it set so that a password is needed for both read and write.

  7. In the DOS window, run ssh:
    You will be prompted to ask if you want to add torte to the authorized hosts, answer is "yes" (all three letters). You should then be connected to torte. You can use it like a telnet session (e.g. try ls, and so on).


Running rsync (Windows 95/98/NT4)

  1. Try rsync:
    This will save the c:\rsync directory to torte, at /rsync/data1/rsync.


  2. For a full drive, e.g. D: try:
    This will save all of D:\* to torte:/rsync/data1/d/*.

    Note: rsync will only create directories one deep. So in the example above, it would create torte:/rsync/data1/d if torte:/rsync/data1 already exists, but not if only torte:/rsync exists (i.e. without data1).

    WARNING!

    Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!


  3. Try one of the batch files to back up a whole drive. I've included two in the package, drivec.bat and drived.bat. These are set to copy the drives to torte:/rsync/data2/scholten -- you should modify them to copy to your own area (e.g. /rsync/data1/username for most people, /rsync/data2/username for atom optics people, /rsync/data3 for Peele). Note that these use the //c notation for Windows NT; you will need to change this to \\c for use on a Win95/98 system.

    You can just execute one of these batch files to mirror your drive. For example, make a shortcut to one of them from an icon on your desktop, and then just click on the shortcut and your whole drive will be mirrored exactly.


  4. Other things to try:

WARNING!

Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!

Recovering files

You can recover files by logging on to torte, and copying the files, or by finding torte in your Network Neighbourhood, or using rsync itself. For example: will list the files in torte:/rsync/data2/scholten. You can recover one with something like: where the . can be replaced with any local destination.

Automating rsync (Windows NT)

This part was really amusing. Not. Jobs run by the NT scheduler are run as user "system", so you have to set things up to run ssh with your personal security keys. Alas, ssh won't even run unless it finds "SYSTEM" in the /etc/passwd file. I now have it working, with "system" running rsync and ssh using my personal security key, normally kept absolutely secret. Here's how to do it.
Note (from Shao-wei (wilson) Hung): Since Internet Explorer 4 , IE comes with Task Scheduler which makes this much easier. So you might want to install IE and then go to the section on automating with Win95/98/2000).
  1. Edit the \etc\passwd file again, adding a second line identical to the first, but change the 1st column to SYSTEM.


  2. Make sure the Schedule service is running. Go to Control Panel, Services, and find Schedule. It will probably have Manual in the startup column. Click on Schedule, then on the Startup... button, and select Automatic and OK. Then click on the Start button and Close.


  3. Open a command window, and type at. It should tell you that there are no entries in the list.


  4. Enter a command like: where 20:00 should be some time in the very near future (e.g. one minute away). When that time comes along, you should see a command window open before your eyes. This command window belongs to user "system".


  5. Change to the rsync directory and test run ssh:
    where username is your username on the remote system (torte). If this works, all is well, and you can proceed. If not, please contact me so I can debug it.

  6. Edit the autotest.bat batch file to set the USERNAME variable to your login name, and also edit the path on torte where you want the files stored.
    Note: I sometimes need to run rsync with -vv or even -vvv for some reason, i.e. lots of debug output, otherwise rsync just does nothing.
  7. Enter a command like: This will run autotest.bat at 8pm every weekday. You can change the time as you wish, and you can repeat this for every time you want rsync to run, and for every drive you want to run. Just run at with no options to see a list of scheduled events. You can delete events as follows: where ID is the job ID (run at for a list).

    Note that the autotest.bat batch file is specially for automated jobs -- it is executed by user "system", but runs rsync and ssh as user USERNAME. Modify it and use it as a basis for any scheduled jobs, e.g. for C drive and so on. Note that you can also run two consecutive rsync jobs from the one batch file, which is better than running two separate scheduled tasks.


    8. Good luck!

Automating rsync (Windows 95/98/2000)

Windows 95/98/2000 (and Internet Explorer) have a GUI "Scheduler" for automating tasks, and jobs are run as the currently logged in user. To add a scheduled task:
  1. click on My Computer and look at the bottom. You should see a folder named Scheduled Tasks
  2. click on Scheduled Tasks
  3. click on Add Scheduled Task
  4. follow the menus; when offered a list of programs to run, select Browse.. and find a batch file such as c:\rsync\drivec.bat
  5. select appropriate times and so on

Running multiple rsyncs simultaneously

Steve Brown discovered that you can only run one rsync at a time, unless you use the -P option for ssh.

Excluding directories

Including and excluding files and directories is well described in the rsync man page. For some good help on this stuff, look at this message by David Bolen.

It can all be very confusing, and two things are not terribly clear. First of all, the basic idea. To exclude various files and directories, you can create a file listing them, and execute rsync with the appropriate option, for example: Let's assume the exclude.txt file looks like: To understand how this works, you should first of all read the man page, and remember:
  1. rsync would compare the source filename "usera" to "+ /usera/". That is, it doesn't compare "/users/usera". The head of the file tree is "/users".
  2. rsync doesn't compare all filenames with the list. That is, it doesn't recursively go into subdirectories once it thinks a directory has been excluded. So in the example above, the "+ /userf" tells rsync to match that subdirectory but not the files and subdirectories in it.
  3. Note that the "- /userb/" and "- /userd/" patterns are redundant; these would be excluded anyway by the final "- /*".

Merging filesets (e.g. laptop+desktop)

This is still experimental. The instructions given here are inefficient in terms of disk use, but safe.

Before you start, it's a good idea to look at the how the files are stored on both machines. For example, if on your laptop you have d:\personal\photos and your desktop has d:\mystuff\photos, then the merged fileset will have both, even if "photos" are the same in both cases. Also, rsync is case sensitive, so if a file is Mydir on one drive and mydir on the other, your merged set will include both. A little effort at homologising your directory structures beforehand can save you lots of time and disk space.

Let's assume you want to merge two filesets, laptop drive D and desktop drive D.
  1. First, make mirrors of the two computers you want to merge, to torte:
  2. Now make a copy of one of these on torte. For example, copy the laptop to a new area which we'll call merge:
  3. Now merge that with the other (i.e. desktop) using rsync: Note the -u option. This says update only so that if a file already exists and is newer, don't overwrite it. Also, the -b option means that old files are backed up (a ~ is added to the backup filename).
  4. Just to be really paranoid, you can do the reverse and see if both merged sets are the same. While still logged on to torte:
  5. At this point you have two hopefully identical merged filesets. You should carefully go through and see if they're what you think they should be. Do search for those files that were backed up (e.g. find /rsync/data1/joeblog/merge -name "*~"). If you're happy with what has happened, you can do it "in real life", for example using rsync to do the merge on your desktop or on your laptop, using torte as the source. Here's how you might want to do it (I haven't tried it yet!). From your laptop: and from your desktop:
Created: 5 January 2000
Updated: 1 February 2000
Maintainer: Robert Scholten, School of Physics, Email:  r.scholten at physics.unimelb.edu.au
Authorised by: Robert Scholten, School of Physics, Email:  r.scholten at physics.unimelb.edu.au
This page, its contents and style, are the responsibility of the author and do not represent the views, policies or opinions of The University of Melbourne.