The rsync provided here is very VERY old. There are many known bugs,
and some of these may cause data corruption. Dan Barrett has alerted me
to one such bug; see here.
Please use one of the alternative Windows-based rsync solutions, for
Installing ssh and rsync on a Windows machine: minimalist approach
Note: again, this stuff is very old. To install OpenSSH on Windows,
This page provides a brief outline of how to get rsync running on a Windows system. This is for the simplest self-contained system without a full Cygwin installation. If you want Cygwin as well, please install Cygwin with openssh and rsync: cygwin.
My version of rsync+ssh is a very stock-standard rsync, with the cygwin1.dll (currently version 1.3.6-6), and a simplified version of ssh. It is version 1.2.26 (yeah, ok, old, but works well enough for this). The main difference is that you can put the whole thing in one directory, wherever you like (e.g. c:\rsync or c:\program files\rsync). And you specify your username and home directory with environment variables (e.g. SET USERNAME=bloggs), i.e. you don't need /etc/passwd. You also don't need /usr, /usr/bin etc.
Binaries are available here. Please do not use
the rsync binaries unless you are willing to corrupt your destination
files. If you are willing to accept that risk, you will find .ZIP and .tar.gz versions of a basic package (rsync, ssh, cygwin1.dll) (734kB) or a full package (1.4MB), and individual files. Note that cygwin1.dll is a completely unaltered version 1.3.6-6.
Source files are here, in .tar.bz2. Note that the cygwin source is a completely unaltered version 1.3.6-6, and the rsync sources are also completely clean. The ssh package is modified a bit to use environment variables instead of an /etc/passwd file.
To use rsync, you first have to have ssh running. Don't even think
about using rsh, and you probably don't want to rely on the built-in
- Unpack rsync_minimal or rsync_full, e.g. as c:\rsync.
Edit the rsetup.bat file. This is a simple batch file that defines your username and home directory and path; modify the appropriate two lines.
Open a DOS or NT command window and go to the appropriate directoy, e.g. cd \rsync
Set the path etc. by running rsetup.
rsync -e ssh -av --delete "/rsync" remote:backupdir
where remote is the name of a remote host (e.g. a Linux box, or optics).
This will save the c:\rsync directory and files to machine remote, at backupdir under your home directory on remote.
For a full drive, e.g. D: try:
rsync -e ssh -av --delete "/cygdrive/d/" remote:backupdir/mypc/d
This will save all of D:\* to remote:backupdir/mypc/d/*.
Note 1: rsync will only create directories one deep. So in the example above, it would create remote:backupdir/mypc/d if remote:backupdir/mypc already exists, but not if only remote:backupdir exists (i.e. without mypc).
Note 2: These examples use a directory relative to your home directory on remote. You can also use absolute directories, e.g. remote:/rsync/data2/blogs.
Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!
Try the example batch file to back up a whole drive, drivec.bat. Edit this to set the appropriate remote hostname and remote directory location.
Note the directory notation, e.g. /cygdrive/c for drive C:\.
You can just execute this batch files to mirror your drive. For
example, make a shortcut to it from an icon on your desktop, and then
just double-click on the shortcut and your whole drive will be
Other things to try:
- -b (i.e. -avb) makes backup copies of any files at the destination, before updating.
- -u (i.e. -avu) updates - won't overwrite newer files.
- -S (i.e. -avS) is more efficient if you have
- rsync -e ssh -av --delete "/cygdrive/d/" remote:/rsync/data1/d >>rsync.log will log the transfers to file rsync.log
- -z (i.e. -avz) is very useful if doing it by modem. This compresses the information before transport. Waste of time if on a fast network.
- -P (i.e. -avP) shows progress during transfers, and also keeps partially transferred files so that if the transfer is interrupted mid-file, the bit already transferred isn't wasted. Again, good if rsyncing by modem.
- The --exclude option is very useful to exclude swap files. You may also want to exclude things like netscape caches. For example:
--exclude 'pagefile.sys' --exclude 'WIN386.SWP' --exclude 'Cache/'
- --backup-dir=PATH puts all old versions of files into the named directory, thus giving an incremental backup.
- Look at the
rsync man page for more ideas.
Be careful of the --delete option. Read the rsync man page for information. It can be catastrophic!
Normally you will have to enter your password every time. However,
you can generate privacy keys for ssh such that you won't
have to enter your password. This is not secure, and if someone gets
the identity key on yorur PC, they can log on to the remote
host as you, and wreak havoc. There is a mechanism for protecting
against this, but for the moment I don't have time to explain. Hence,
use this at your own risk.
You can recover files by logging on to remote, and copying
the files, or by finding remote in your Network Neighbourhood, or using rsync itself. For example:
Generate your security keys by running ssh-keygen:
You will be asked Enter file in which to save the key (/rsync/.ssh/identity):, just hit Enter.
WARNING: This creates a ssh key without a passphrase. That means that anyone can use your Win95/98 machine to connect to any of the hosts that you set up, without having to type a password ever (because Win95/98 don't have real passwords). If anyone else has access to your PC, then use ssh-keygen and enter a passphrase when prompted.
- Add \rsync\.ssh\identity.pub to the end of your
~/.ssh/authorized_keys file on the system you want to send
your files to. Let's assume this is a machine called remote.
Copy identity.pub to remote with
a command like:
copy \rsync\.ssh\identity.pub \\remote\myusername\.ssh
where you need to replace myusername with your username.
Then log in to remote, and:
cat identity.pub >>.ssh/authorized_keys
WARNING: Once this is done, anyone that has a copy of your private key, \rsync\.ssh\identity (not your public key, identity.pub) can log in to remote without a password. So, make sure you don't let anyone read this file. In particular, make sure that if you "share" your drive, you have it set so that a password is needed for both read and write.
In the DOS or CMD window, run ssh:
You will be prompted to ask if you want to add remote to the authorized hosts, answer is "yes" (all three letters).
You should then be connected to remote. You can use it like a telnet session (e.g. try ls, and so on).
rsync -e ssh -a remote:/rsync/data1/d
will list the files in remote:/rsync/data1/d. You can recover one with something like:
rsync -e ssh -a remote:/rsync/data1/d/testfile.txt .
where the . can be replaced with any local destination.
This is most easily done with the Windows Task Scheduler.
If you have an old Windows, you can get Task Scheduler by downloading the free Internet Explorer (version 4 or later).
To add a scheduled task:
Steve Brown discovered that you can only run one rsync at a time, unless you use the -P option for ssh.
Including and excluding files and directories is well described in the rsync man page. For some good help on this stuff, look at this message by David Bolen.
- click on My Computer and look at the bottom. You should see a folder named Scheduled Tasks
- click on Scheduled Tasks
- click on Add Scheduled Task
- follow the menus; when offered a list of programs to run, select Browse.. and find a batch file such as c:\rsync\drivec.bat
- select appropriate times and so on
WARNING! If doing this on a PC, be careful that your
--exclude-from or --include-from file is in Unix format (newlines
rather that cr/lf). You have been warned! Put a dos2ux line
in your rsync script to ensure this. Again, you have been warned!
WARNING! Make sure source directory ends with a / (in this
case, /users/), and make sure destination directory does not
(in this case /rsync/users).
It can all be very confusing, and two things are not terribly clear.
First of all, the basic idea. To exclude various files and
directories, you can create a file listing them, and execute rsync
with the appropriate option, for example:
rsync -e ssh -axv --delete --exclude-from=exclude.txt /users/ remote:/rsync/users
Let's assume the exclude.txt file looks like:
To understand how this works, you should first of all read the man page, and remember:
- rsync would compare the source filename "usera" to "+ /usera/". That is, it doesn't compare "/users/usera". The head of the file tree is "/users".
- rsync doesn't compare all filenames with the list. That is, it doesn't recursively go into subdirectories once it thinks a directory has been excluded. So in the example above, the "+ /userf" tells rsync to match that subdirectory but not the files and subdirectories in it.
- Note that the "- /userb/" and "- /userd/" patterns are redundant; these would be excluded anyway by the final "- /*".
This is still experimental. The instructions given here are inefficient in terms of disk use, but safe.
- Include/Exclude file lists
On windows, your
--exclude-from or --include-from file must be in
Unix format (newlines rather that cr/lf). Put a dos2ux line
in your rsync script to ensure this.
Windows doesn't keep file modification times to
better than two seconds. Use the --modify-window=2 option to
get around this.
- Time zones
rsync tries to convert all file timestamps
into UTC at both ends. If seemingly identical files with the same
timestamp seem to be transferring, try this at both ends:
If the remote and local files have the same timestamp, and they're
still transferring, then you have a different problem. Otherwise, try
--modify-window=3601 which means ignore time differences of
less than an hour. This can be very helpful after a daylight savings
ls -l --full-time file_in_question
- cygwin using ascii mode to read files
decides to open files in ascii format (i.e. convert newlines to cr/lf
pairs). If this occurs, you need to mount your local drive (the one
with the files you want to rsync) in binmode. This seems to be a bit
unreliable but as far as I can tell, this is supposed to work. Try
one of the following (the -s option adds the mount to the cygwin
section of the registry, so it should "stick").
mount -b -f -s "d:/" "/cygdrive/d"
mount -b -f "d:/" "/cygdrive/d"
Before you start, it's a good idea to look at the how the files are stored on both machines. For example, if on your laptop you have d:\personal\photos and your desktop has d:\mystuff\photos, then the merged fileset will have both, even if "photos" are the same in both cases. Also, rsync is case sensitive, so if a file is Mydir on one drive and mydir on the other, your merged set will include both. A little effort at homologising your directory structures beforehand can save you lots of time and disk space.
Let's assume you want to merge two filesets, laptop drive D and desktop drive D.
- First, make mirrors of the two computers you want to merge, to a server, e.g. a computer called remote:
rsync -e ssh -avbu --delete "\\d/" remote:/rsync/data1/joeblog/laptop/d
rsync -e ssh -avbu --delete "\\d/" remote:/rsync/data1/joeblog/desktop/d
- Now make a copy of one of these on remote. For example, copy the laptop to a new area which we'll call merge:
- ssh remote
- cd /rsync/data1/joeblog
- mkdir merge
- cp -a laptop/d merge (might take awhile!)
- Now merge that with the other (i.e. desktop) using rsync:
Note the -u option. This says update only so that if a file already exists and is newer, don't overwrite it. Also, the -b option means that old files are backed up (a ~ is added to the backup filename).
- rsync -auvb /rsync/data1/joeblog/desktop/d/ /rsync/data1/joeblog/merge/d
- Just to be really paranoid, you can do the reverse and see if both merged sets are the same. While still logged on to remote:
- cd /rsync/data1/joeblog
- mkdir merge2
- cp -a desktop/d merge2 (again, might take a long time)
- rsync -auvb /rsync/data1/joeblog/laptop/d/ /rsync/data1/joeblog/merge2/d
- At this point you have two hopefully identical merged filesets. You should carefully go through and see if they're what you think they should be. Do search for those files that were backed up (e.g. find /rsync/data1/joeblog/merge -name "*~"). If you're happy with what has happened, you can do it "in real life", for example using rsync to do the merge on your desktop or on your laptop, using host remote as the source. Here's how you might want to do it (I haven't tried it yet!). From your laptop:
and from your desktop:
- rsync -auv remote:/rsync/data1/joeblog/desktop/d/ "\\d/"
- rsync -auv remote:/rsync/data1/joeblog/laptop/d/ "\\d/"
Created: 5 January 2000
Updated: 21 April 2001
Maintainer: Robert Scholten, School of Physics, Email: r.scholten at physics.unimelb.edu.au
Authorised by: Robert Scholten, School of Physics, Email: r.scholten at physics.unimelb.edu.au
This page, its contents and style, are the responsibility of the author and do not represent the views, policies or opinions of The University of Melbourne.