How to Synchronize Files on Multiplatforms

Synchronize Files

Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.
Unison shares a number of features with tools such as configuration management packages (CVS, PRCS, Subversion, BitKeeper, etc.), distributed filesystems (Coda, etc.), uni-directional mirroring utilities (rsync, etc.), and other synchronizers (Intellisync, Reconcile, etc). However, there are several points where it differs:

• Unison runs on both Windows and many flavors of Unix (Solaris, Linux, OS X, etc.) systems. Moreover, Unison works across platforms, allowing you to synchronize a Windows laptop with a Unix server, for example.
• Unlike simple mirroring or backup utilities, Unison can deal with updates to both replicas of a distributed directory structure. Updates that do not conflict are propagated automatically. Conflicting updates are detected and displayed.
• Unlike a distributed filesystem, Unison is a user-level program: there is no need to modify the kernel or to have superuser privileges on either host.
• Unison works between any pair of machines connected to the internet, communicating over either a direct socket link or tunneling over an encrypted ssh connection. It is careful with network bandwidth, and runs well over slow links such as PPP connections. Transfers of small updates to large files are optimized using a compression protocol similar to rsync.
• Unison is resilient to failure. It is careful to leave the replicas and its own private structures in a sensible state at all times, even in case of abnormal termination or communication failures.
• Unison has a clear and precise specification.
• Unison is free; full source code is available under the GNU Public License.

Install Unison on Linux

To install Unison on Debian, Linux or Linux Mint:

$ sudo apt-get install unison

Create Unison Profile

The first thing to do is to create a Unison profile on both servers which have replicas to sync. A Unison profile is a text file (with .prf extension) that specifies file sync settings such as directory roots, include/ignore paths or patterns, etc.

You can create a Unison profile anywhere in your system, in which case you must define UNISON environment variable pointing to the directory path to the profile. If UNISON variable is not defined, Unison looks for profiles in $HOME/.unison directory by default.

The following snippet is a Unison profile example.

# Two root directories to sync.
# You can use ssh:// to sync over SSH
root = /home/alice/sync_folder
root = ssh://dev@192.168.1.10//home/cyberpunk/sync_folder

# If you want one-way mirroring from one replica to the other, specify the source replica using "force" as follows.
# force = /home/cyberpunk/sync_folder

# If you want Unison to run without any user input, try "batch" mode.
batch = true

# If you don't want to be prompted, and just accept Unison's recommendation:
auto = true

# Optionally, you can sync specific sub directories only (under the root).
# path = dir1
# path = dir2

# Optionally, you can ignore specific files or directories that are matched with regular expressions.
# ignore = Name *.o
# ignore = Name *~
# ignore = Path */temp/archive_*

# If you want to ignore difference in file props:
perms = 0

The most important information in the above are the two root directories to sync. Unison supports SSH, RSH or socket to sync files over network. So you can specify the root directory on a remote host by using one of the formats below:

SSH:

root = ssh://user@remote_host//absolute/path/to/root
root = ssh://user@remote_host/relative/path/to/root

RSH:

root = rsh://user@remote_host//absolute/path/to/root
root = rsh://user@remote_host/relative/path/to/root

Socket:

socket://remote_host:port_num//absolute/path/to/root
socket://remote_host:port_num/relative/path/to/root

Sync Files with Unison

Once a Unison profile has been created on both servers, simply run Unison from either server.

$ unison

If there are multiple Unison profiles (e.g., sync1.prf, sync2.prf) in Unison directory, you can specify the profile name as follows.

$ unison sync1

If you have not set up key-based SSH authentication for the remote host, Unison will ask you to log in to the remote host via SSH before performing file sync. If you have set up passwordless SSH login, Unison will automatically start file sync over SSH.

Contacting server...
Connected [//local//home/cyberpunk/sync_folder -> //remote_host//home/cyberpunk/sync_folder]
Looking for changes
Waiting for changes from server
Reconciling changes
new file --> document1.pdf
<-- new file my.jpg
Propagating updates
UNISON 2.40.63 started propagating changes at 21:19:13.65 on 20 Sep 2013
[BGN] Copying document1.pdf from /home/cyberpunk/sync_folder to //remote_host//home/cyberpunk/sync_folder
[BGN] Copying my.jpg from //remote_host//home/cyberpunk/sync_folder to /home/cyberpunk/sync_folder
[END] Copying my.jpg
[END] Copying document1.pdf
UNISON 2.40.63 finished propagating changes at 21:19:13.68 on 20 Sep 2013
Saving synchronizer state
Synchronization complete at 21:19:13 (2 items transferred, 0 skipped, 0 failed)

More information can be found at: http://www.cis.upenn.edu/~bcpierce/unison/download.html and at: http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html