Tuesday 1st January 2008

Frank Chant

My friend Frank Chant died alone in his flat today. His television was on and his newspaper was outside his door, so his landlord went in to investigate and discovered the body. The cause of death isn't yet known.

I've known Frank for over 20 years and we went to many football matches together over the years, most recently the Liverpool game two days ago, where he looked as fit, healthy, and happy as ever I'd seen him.

His friendship, warmth, and humour will be sorely missed.

Wednesday 2nd January 2008

Fedora Project

• Updated gtkwave to 3.1.2 in Rawhide

• Raised Bug #427274 on logrotate, which in Fedora 8 isn't preserving the SELinux file contexts for the files it rotates, such as /var/log/btmp and /var/log/rpmpkgs, which results in SELinux denials like these:

• type=AVC msg=audit(1199231495.982:35629): avc:  denied  { append } for pid=30802 comm="sshd" name="btmp" dev=dm-3 ino=1212485 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:var_log_t:s0 tclass=file

Local Packages

• Updated dovecot to 1.0.10

• Updated gtkwave to 3.1.2

• Updated nmap to 4.52

• Updated perl-Net-DNS to 0.62

Thursday 3rd January 2008

Local Packages

• Updated mod_fcgid to update the SELinux policy to do temporary file transitions to httpd_tmp_t, which fixes a problem with uploading attachments in moin

• Updated nmap to use pathname-based build dependencies now that the mock bug preventing them from being resolved properly has finally been fixed in mock 0.8.19.

Fedora Project

• Updated mod_fcgid for Fedora releases 7 and 8, plus Rawhide, to include the SELinux policy fix

Friday 4th January 2008

Fedora Project

• Updated python-zope-interface to build on Rawhide, where python egg info files are now built automatically; I'm going to need to do something similar for bittorrent but there are a number of outstanding bugs that need fixing there, which might as well get rolled up into one big update

Local Packages

• Updated python-zope-interface as per the Fedora package

• Updated php4-pcntl to 4.4.8

• Rebuilt php4-pcntl-gtk against the updated php4-pcntl

Friday 11th January 2008

Local Packages

• Updated libidn to 1.4

• Rebuilt dovecot in line with Rawhide version

• Rebuilt nmap to merge in changes from Rawhide, which is now at the same version; the zenmap package is now renamed back to nmap-frontend

• Updated curl to merge patch from Rawhide for Bug #427966 (do not attempt to close a bad socket)

Monday 14th January 2008

Local Packages

• Updated mod_fcgid to fix occasional crashes at restarts with SELinux in enforcing mode

• Updated nmap to 4.53

Fedora Project

• Fixed mod_fcgid as per the local package and built updates for Rawhide, F-8, and F-7

Tuesday 15th January 2008

Local Packages

• Updated gtkwave to 3.1.3

Fedora Project

• Updated gtkwave to 3.1.3 in Rawhide

Monday 21st January 2008

Local Packages

• Updated perl-BerkeleyDB to 0.33

This proved to be troublesome to build on older distributions such as Fedora Core 2 and older, and CentOS 3 (i386). There seemed to be lots of problems with mutexes though strangely the x86_64 version for CentOS 3 was OK. Looking through the README file included in the upstream distribution suggested that Red Hat's packaging was a problem, and included a reference to Bug #91933. Following the trail, it appeared to be an issue with the i386 version of glibc on those distributions. Looking at the root.log file produced by mock for these builds, I could see that the i386 version of glibc was indeed being pulled in, as opposed to say the i586, i686, or athlon versions. So, I tried adding the following to the yum configuration for mock for the problematic releases:

# We don't want the i386 glibc because it doesn't support NTPL threads properly
# and causes build failures for perl-BerkeleyDB
exclude=glibc.i386

This resulted in i586, i686, or athlon versions of glibc being installed into the buildroot by mock, and the build problems went away

Fedora Project

• Updated bluefish in Fedora 8 and Rawhide to include upstream patch for Bug #390871 (Unable to change syntax highlighting in Bluefish)

Tuesday 22nd January 2008

Local Packages

• Updated curl to fix obsoletes on curl-devel and ensure a sane upgrade path from Fedora 8

Wednesday 23rd January 2008

Local Packages

• Applied the patch from CPAN RT#29593 to perl-HTML-Tidy, which enables it to build successfully against the latest libtidy

Monday 28th January 2008

Local Packages

• Updated perl-IO-Socket-SSL to 1.13

Tuesday 29th January 2008

Local Packages

• Updated curl to 7.18.0

Thursday 31st January 2008

Local Packages

• Updated moin to 1.6.0

• Updated moin-theme-monobook to make it work with moin 1.6.0; unfortunately that makes it incompatible with older versions of moin

Wiki Upgrade

Upgraded this wiki to moin 1.6.0; this was a decidedly non-trivial upgrade, which I'll describe here. For reference, this wiki runs using mod_fcgid on Apache httpd, and the files live under /srv/www/tips.

The first step was to turn off access to the wiki from the network so that the data wouldn't change underneath us. I did this by creating a static wiki unavailable page and adding a redirect for the wiki location on the web server to serve out the temporary static page instead:

    # Temporarily make the wiki unavailable
    RedirectMatch seeother ^/tips/.*           /wiki-unavailable.html

After setting this up and reloading httpd, all accesses to the wiki got the wiki unavailable page. So far so good.

Next step was install the new CGI:

# cd /srv/www/tips
# cp /usr/share/moin/server/moin.fcg cgi-bin/moin.fcgi
# chcon system_u:object_r:httpd_fastcgi_script_exec_t:s0 cgi-bin/moin.fcgi

I then edited /srv/www/tips/cgi-bin/moin.fcgi as follows:

--- cgi-bin/moin.fcgi.160        2008-01-31 11:29:24.000000000 +0000
+++ cgi-bin/moin.fcgi            2008-01-31 11:30:18.000000000 +0000
@@ -15,7 +15,7 @@

 # Path of the directory where wikiconfig.py is located.
 # YOU NEED TO CHANGE THIS TO MATCH YOUR SETUP.
-sys.path.insert(0, '/path/to/wikiconfig')
+sys.path.insert(0, '.')

 # Path of the directory where farmconfig is located (if different).
 #sys.path.insert(0, '/path/to/farmconfig')
@@ -28,7 +28,7 @@

 class Config(FastCgiConfig):
     #loglevel_file = logging.DEBUG  # adapt if you don't like the default
-    logPath = 'moin.log'
+    logPath = '/var/log/moin/tips.log'

     properties = {}
     # properties = {'script_name': '/'} # use this instead of the line above if your wiki runs under "/" url

As moin now writes to its own log files, I had to set up a directory for it to do this:

# mkdir /var/log/moin
# chown apache:apache /var/log/moin
# chmod 755 /var/log/moin
# chcon -t httpd_fastcgi_script_rw_t /var/log/moin
# semanage fcontext -a -t httpd_fastcgi_script_rw_t '/var/log/moin(/.*)?'

As the wikiconfig.py has changed quite a lot from 1.5.x to 1.6.0, I decided it would be cleaner to start with a pristine 1.6.0 wikiconfig.py and customize that rather than tweak the existing wikiconfig.py. The changes I made were:

sitename = u'Hints, Tips and HOWTOs'
logo_string = sitename
page_front_page = u"FrontPage"
interwikiname = 'Tips'
data_dir = '/srv/www/tips/data/'
data_underlay_dir = '/srv/www/tips/underlay/'
superuser = [u"PaulHowarth", ]
acl_rights_before  = u"PaulHowarth:admin,read,write,delete,revert"
acl_rights_default = u"AdminGroup:admin,read,write,delete,revert All:read"
mail_smarthost = "localhost"
mail_from = "Hints Tips and HOWTOs Wiki <hints-tips-howtos@city-fan.org>"
navi_bar = [
    u'%(page_front_page)s',
    u'SiteNavigation',
    u'TitleIndex',
    u'RecentChanges',
    u'FindPage',
    u'HelpContents',
]
theme_default = 'monobook'
edit_locking = 'lock 10'
shared_intermap = '/srv/www/tips/data/local-intermap.txt'

Some tweaking of the httpd configuration was needed too, to handle the new URL prefix for static data:

Alias /moin_static160 "/usr/share/moin/htdocs/"

<Directory "/usr/share/moin/htdocs/">
    Options Indexes FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
    <IfModule mod_expires.c>
         ExpiresActive On
         ExpiresDefault "access plus 1 year"
    </IfModule>
</Directory>

The next step was to run the upstream migration script for the content. This runs as user apache and wants to create a backup data directory under /srv/www/tips, so I had to temporarily open up the permissions to do this:

# chmod 777 /srv/www/tips
# runuser -s /bin/sh -c "moin --config-dir=/srv/www/tips/cgi-bin --wiki-url=www.city-fan.org/tips/ migration data" apache
# chmod 755 /srv/www/tips

I also needed to update the monobook theme to the patched version that works with moin 1.6.0 (already installed via the RPM):

# cd data/plugin/theme
# rm monobook.py*
# cp /usr/share/moin/data/plugin/theme/monobook.py* .
# chown apache:apache monobook.py*
# cd -

The migrated data pages lost their security conexts, so I fixed them up and updated the underlay pages as usual:

# chcon -R -t httpd_fastcgi_script_rw_t data
# mv underlay underlay.old
# cp -a /usr/share/moin/underlay .
# chown -R apache:apache underlay
# chcon -R -t httpd_fastcgi_script_rw_t underlay

I then turned off the temporary redirection of wiki accesses, and restarted httpd. It sort of worked but it was apparent that the migration of the pages to handle the new wiki markup was less than complete.

The first thing I noticed was that most of the code examples I use were broken. I tend to use markup like this

{{{# /path/to/example/command --args
}}}

That doesn't render the same in moin 1.6.x, and had to be changed to:

{{{
# /path/to/example/command --args
}}}

This problem was quite easy to fix up with a one-liner:

# find data/pages -type f -name '[0-9]*' | xargs sed -i -e 's/{{{\(.\)/{{{\n\1/g'

The other problem I noticed was that a large proportion of the InterWiki links were broken. For instance, all the ones that looked like this:

[wiki:RedHatBugzilla:123456 Bug #123456]

had been rewritten as:

[[RedHatBugzilla:/InterWiki|Bug #123456]]

instead of:

[[RedHatBugzilla:123456|Bug #123456]]

A quick grep revealed that there were a lot of instances of this. Many of them were in the same standard format though, which would lend itself to being fixed by a script. First I put together a script that would print out the filename of the current revision of each page, which I saved as /srv/www/tips/pagelist:

#!/bin/bash

cd /srv/www/tips/data/pages
for page in $(find . -mindepth 1 -maxdepth 1 -type d | sed -e 's,^./,,' | sort); do
        index_file="${page}/current"
        if [ ! -f "${index_file}" ]; then
                echo "Index file missing for ${page}" 1>&2
                continue
        fi
        current_rev="${page}/revisions/$(cat ${index_file})"
        echo "${current_rev}"
done

Using this script I was then able to fix a lot of the bugzilla references:

# cd data/pages
# ../../pagelist 2>/dev/null | xargs sed -i -e 's@\[\[RedHatBugzilla:/InterWiki[|]Bug #\([0-9]*\)\]\]@[[RedHatBugzilla:\1|Bug #\1]]@g'
# cd -

That left over a hundred broken links of various types, on the pages listed by this command:

# fgrep /InterWiki `./pagelist 2>/dev/null | awk '{ print "data/pages/" $0 }'` | sed 's/:.*//' | uniq

I ended up fixing these manually using the wiki editor, using the previous page version as a reference. That was painful, but, having done it, everything then seemed OK.