Table of Contents

Dokuwiki & rsync

A method to synchronize two DokuWiki installations.

For more elaborate concepts on this topic see dokupubsub.

About

The idea is simple: For each wiki-write-transaction a small DokuWiki action plugin launches rsync to publish/update the underlying data storage.

Synchronization happens in the background: a small perl script prevents multiple simultaneous rsync processes and re-runs if changes occur during an active transfer.

Notes

Note: The index of the receiving side is not updated automatically.

While rsync is capable of bi-directionally synchronizing multiple wikis. The script below is in use on a (read/write) master to (mostly read-only) slave wiki. Use rsync's -u option, but with care: page-locks, and edit-drafts are not distributed. - While suitable for a small group of editors, race-conditions may occur on larger public wikis.

One workaround is to make a receiving namespace read-only; optionally in combination with the include plugin to make parts locally writable content.

Alternatively one can make use of redirecting or proxying write-requests (HTTP-POSTs in general) to the central master. There can be different master servers per namespace; but depends on secure cross-site requests and authentication token passing.

Last but not least NFS-sharing the data/lock folder may be an option. see dokupubsub.

http://dokuwiki.org/tips:farm and http://dokuwiki.org/tips:farm2 have hints about patching DokuWiki to set up master server (aka. farmer).

Source

Configure and test the perl script first; then save this plugin as lib/plugins/rsync/action.php and edit a page to test it. check /tmp/rwikisync.log.

<?php
/**
 * @license    GPL 2 (http://www.gnu.org/licenses/gpl.html)
 * @author     Robin Gareus <robin@gareus.org>
 */
 
// must be run within Dokuwiki
if(!defined('DOKU_INC')) die();
 
if(!defined('DOKU_PLUGIN')) define('DOKU_PLUGIN',DOKU_INC.'lib/plugins/');
  require_once (DOKU_PLUGIN.'action.php');
 
class action_plugin_rsync extends DokuWiki_Action_Plugin {
 
  function getInfo(){
    return array(
      'author' => 'Robin Gareus',
      'email'  => 'robin@gareus.org',
      'date'   => '2007-05-23',
      'name'   => 'rsync plugin',
      'desc'   => 'rsync dokuwiki data',
      'url'    => '',
    );
  }
 
  function register($contr){
    $contr->register_hook('IO_WIKIPAGE_WRITE',
                          'AFTER',
                          $this,
                          'handle_wikipage_write',
                          array());
  }
 
  function handle_wikipage_write($event, $param) {
    // if an old revision is saved or the file is empty -> run away
    if ( $event->data[3] || !$event->data[0][1] ) return false;
    //$ns=$event->data[1];
    //$path=$event->data[0][0];
 
    system ("/usr/local/bin/dokusync.pl &>/tmp/rwikisync.log &");
    return true;
  }
 
}

The Perl script to rsync the actual data. Save as /usr/local/bin/dokusync.pl or change the path in the above plugin code accordingly.

You need to change username and provide a ssh-key (just generate one with ssh-keygen). There's many tutorials on ssh keys and rsync.

#!/usr/bin/perl
 
### CONFIG
$rsyncopts="-azv --delete --no-o --no-g --chmod=Dg+wx,ug+rw";
$myssh="ssh -i /var/opt/id_rsa -l username";
@excludelist  = ( "_darcs", "cache/", "attic/", "private/", "index/" );
$src="/var/www/data/";
$dstpath="/site/docroot/data/";
$dsthost="example.org";
 
$dst=$dsthost.":".$dstpath;
 
###prevent multiple instances
use File::Pid;
$pidfile = File::Pid->new({ file => "/var/lock/rwikisync.pid"});
if ( my $num = $pidfile->running ) {
        # touch restart-file
        system("touch /var/lock/rsyncrestart");
        die "Already running: $num - restart scheduled\n";
}
$pidfile->write;
 
### build commandline
$command="rsync $rsyncopts -e \"$myssh\" ";
foreach (@excludelist) {
  $command.=" --exclude '$_'";
}
$command.=" $src $dst";
### and execute it..
system("date");
system($command);
### fix group permissions
system("$myssh $dsthost \"chgrp -R www-data $dstpath 2>/dev/null\"");
 
# check if restart was attempted -> rm restart file and rsync again..
while ( -e "/var/lock/rsyncrestart" ) {
        unlink "/var/lock/rsyncrestart";
        system($command);
}
 
$pidfile->remove;
 
wiki/dokursync.txt · Last modified: 13.08.2010 02:14 by 65.203.131.194
   |