PaulHowarth/Blog/2008-01-31

Thursday 31st January 2008

Local Packages

  • Updated moin to 1.6.0

  • Updated moin-theme-monobook to make it work with moin 1.6.0; unfortunately that makes it incompatible with older versions of moin

Wiki Upgrade

Upgraded this wiki to moin 1.6.0; this was a decidedly non-trivial upgrade, which I'll describe here. For reference, this wiki runs using mod_fcgid on Apache httpd, and the files live under /srv/www/tips.

The first step was to turn off access to the wiki from the network so that the data wouldn't change underneath us. I did this by creating a static wiki unavailable page and adding a redirect for the wiki location on the web server to serve out the temporary static page instead: 

    # Temporarily make the wiki unavailable
    RedirectMatch seeother ^/tips/.*           /wiki-unavailable.html

After setting this up and reloading httpd, all accesses to the wiki got the wiki unavailable page. So far so good.

Next step was install the new CGI: 

# cd /srv/www/tips
# cp /usr/share/moin/server/moin.fcg cgi-bin/moin.fcgi
# chcon system_u:object_r:httpd_fastcgi_script_exec_t:s0 cgi-bin/moin.fcgi

I then edited /srv/www/tips/cgi-bin/moin.fcgi as follows: 

--- cgi-bin/moin.fcgi.160        2008-01-31 11:29:24.000000000 +0000
+++ cgi-bin/moin.fcgi            2008-01-31 11:30:18.000000000 +0000
@@ -15,7 +15,7 @@

 # Path of the directory where wikiconfig.py is located.
 # YOU NEED TO CHANGE THIS TO MATCH YOUR SETUP.
-sys.path.insert(0, '/path/to/wikiconfig')
+sys.path.insert(0, '.')

 # Path of the directory where farmconfig is located (if different).
 #sys.path.insert(0, '/path/to/farmconfig')
@@ -28,7 +28,7 @@

 class Config(FastCgiConfig):
     #loglevel_file = logging.DEBUG  # adapt if you don't like the default
-    logPath = 'moin.log'
+    logPath = '/var/log/moin/tips.log'

     properties = {}
     # properties = {'script_name': '/'} # use this instead of the line above if your wiki runs under "/" url

As moin now writes to its own log files, I had to set up a directory for it to do this: 

# mkdir /var/log/moin
# chown apache:apache /var/log/moin
# chmod 755 /var/log/moin
# chcon -t httpd_fastcgi_script_rw_t /var/log/moin
# semanage fcontext -a -t httpd_fastcgi_script_rw_t '/var/log/moin(/.*)?'

As the wikiconfig.py has changed quite a lot from 1.5.x to 1.6.0, I decided it would be cleaner to start with a pristine 1.6.0 wikiconfig.py and customize that rather than tweak the existing wikiconfig.py. The changes I made were: 

sitename = u'Hints, Tips and HOWTOs'
logo_string = sitename
page_front_page = u"FrontPage"
interwikiname = 'Tips'
data_dir = '/srv/www/tips/data/'
data_underlay_dir = '/srv/www/tips/underlay/'
superuser = [u"PaulHowarth", ]
acl_rights_before  = u"PaulHowarth:admin,read,write,delete,revert"
acl_rights_default = u"AdminGroup:admin,read,write,delete,revert All:read"
mail_smarthost = "localhost"
mail_from = "Hints Tips and HOWTOs Wiki <hints-tips-howtos@city-fan.org>"
navi_bar = [
    u'%(page_front_page)s',
    u'SiteNavigation',
    u'TitleIndex',
    u'RecentChanges',
    u'FindPage',
    u'HelpContents',
]
theme_default = 'monobook'
edit_locking = 'lock 10'
shared_intermap = '/srv/www/tips/data/local-intermap.txt'

Some tweaking of the httpd configuration was needed too, to handle the new URL prefix for static data: 

Alias /moin_static160 "/usr/share/moin/htdocs/"

<Directory "/usr/share/moin/htdocs/">
    Options Indexes FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
    <IfModule mod_expires.c>
         ExpiresActive On
         ExpiresDefault "access plus 1 year"
    </IfModule>
</Directory>

The next step was to run the upstream migration script for the content. This runs as user apache and wants to create a backup data directory under /srv/www/tips, so I had to temporarily open up the permissions to do this: 

# chmod 777 /srv/www/tips
# runuser -s /bin/sh -c "moin --config-dir=/srv/www/tips/cgi-bin --wiki-url=www.city-fan.org/tips/ migration data" apache
# chmod 755 /srv/www/tips

I also needed to update the monobook theme to the patched version that works with moin 1.6.0 (already installed via the RPM): 

# cd data/plugin/theme
# rm monobook.py*
# cp /usr/share/moin/data/plugin/theme/monobook.py* .
# chown apache:apache monobook.py*
# cd -

The migrated data pages lost their security conexts, so I fixed them up and updated the underlay pages as usual: 

# chcon -R -t httpd_fastcgi_script_rw_t data
# mv underlay underlay.old
# cp -a /usr/share/moin/underlay .
# chown -R apache:apache underlay
# chcon -R -t httpd_fastcgi_script_rw_t underlay

I then turned off the temporary redirection of wiki accesses, and restarted httpd. It sort of worked but it was apparent that the migration of the pages to handle the new wiki markup was less than complete.

The first thing I noticed was that most of the code examples I use were broken. I tend to use markup like this 

{{{# /path/to/example/command --args
}}}

That doesn't render the same in moin 1.6.x, and had to be changed to: 

{{{
# /path/to/example/command --args
}}}

This problem was quite easy to fix up with a one-liner: 

# find data/pages -type f -name '[0-9]*' | xargs sed -i -e 's/{{{\(.\)/{{{\n\1/g'

The other problem I noticed was that a large proportion of the InterWiki links were broken. For instance, all the ones that looked like this: 

[wiki:RedHatBugzilla:123456 Bug #123456]

had been rewritten as: 

[[RedHatBugzilla:/InterWiki|Bug #123456]]

instead of: 

[[RedHatBugzilla:123456|Bug #123456]]

A quick grep revealed that there were a lot of instances of this. Many of them were in the same standard format though, which would lend itself to being fixed by a script. First I put together a script that would print out the filename of the current revision of each page, which I saved as /srv/www/tips/pagelist: 

#!/bin/bash

cd /srv/www/tips/data/pages
for page in $(find . -mindepth 1 -maxdepth 1 -type d | sed -e 's,^./,,' | sort); do
        index_file="${page}/current"
        if [ ! -f "${index_file}" ]; then
                echo "Index file missing for ${page}" 1>&2
                continue
        fi
        current_rev="${page}/revisions/$(cat ${index_file})"
        echo "${current_rev}"
done

Using this script I was then able to fix a lot of the bugzilla references: 

# cd data/pages
# ../../pagelist 2>/dev/null | xargs sed -i -e 's@\[\[RedHatBugzilla:/InterWiki[|]Bug #\([0-9]*\)\]\]@[[RedHatBugzilla:\1|Bug #\1]]@g'
# cd -

That left over a hundred broken links of various types, on the pages listed by this command: 

# fgrep /InterWiki `./pagelist 2>/dev/null | awk '{ print "data/pages/" $0 }'` | sed 's/:.*//' | uniq

I ended up fixing these manually using the wiki editor, using the previous page version as a reference. That was painful, but, having done it, everything then seemed OK.

Recent