Paul's Blog Entries for June 2007

Friday 1st June 2007

Local Packages

Fedora Project

Monday 4th June 2007

Fedora Project

Local Packages

I set up my buildsystem to create packages for Fedora 7, and to label Rawhide packages with the fc8 tag. I've tweaked my buildsys-build package to not require buildsys-macros for Fedora 7 onwards since the required rpm macros are now included in redhat-rpm-config

My biggest concern was what to do with the merged Core and Extras repository. For all previous releases, I used a loopback-mounted DVD ISO image as a local repository for the base OS, and grabbed extras packages over the network via a local squid cache (much as is described in PackageMaintainers/MockTricks).

In order to build arbitrary packages for Fedora 7, I need the Everything repository (the full merged Core and Extras set) in the mock configuration. However, I also want to speed things up by using packages from my local DVD image where possible. The difficulty here is that the DVD image contains the same packages as the Everything repository, and there's no documented way to get yum to pick packages from the DVD ISO first and only go to the network if the package isn't found there.

It does appear though that yum currently searches through repositories in order of repository ID (I realise that this may change without warning at any given moment...) so the following mock config is serving my needs for x86_64:

$ cat /etc/mock/fedora-7-x86_64.cfg 
#!/usr/bin/python -tt
import os

config_opts['root'] = 'fedora-7-x86_64'
config_opts['target_arch'] = 'x86_64'

config_opts['yum.conf'] = """
[main]
cachedir=/var/cache/yum
debuglevel=1
reposdir=/dev/null
logfile=/var/log/yum.log
retries=20
obsoletes=1
gpgcheck=0
assumeyes=1
exclude=[ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefhijklmnopqrstuvwxyz]*.i*86 g[abcdefghijkmnopqrstuvwxyz]*.i?86 glib2.i?86 glib.i?86 *-devel.i?86

# repos

[zzz-everything]
name=zzz-everything
baseurl=http://sunsite.mff.cuni.cz/MIRRORS/fedora.redhat.com/fedora/7/Everything/x86_64/os
proxy=http://localhost:3128/
priority=1
protect=1

[local-media]
name=local-media
baseurl=file:///srv/nb/distros/fc7/os/x86_64

[updates-released]
name=updates
baseurl=file:///srv/nb/distros/fc7/updates/x86_64

[groups]
name=groups
baseurl=file:///srv/nb/distros/fc7/groups

[city-fan.org]
name=city-fan.org repository for Fedora 7 (x86_64)
baseurl=file:///home/paul/cfo-repo/yum-repo/fc7/x86_64
enabled=0

"""

(the priority and protect lines were copied from a similar arrangement that CentOS uses, but doesn't seem to have any effect)

Wednesday 6th June 2007

Local Packages

Thursday 7th June 2007

Local Packages

I also had a hack at the repoclosure script from yum-utils so that I could check the repositories for each distribution much more quickly. The standard repoclosure script checks all of the packages in all of the enabled repositories to make sure that their dependencies can be satisfied. My repositories are small relative to the main distribution repositories, and the majority of the time it takes repoclosure to run is spent checking the main distribution repositories for dependency closure against themselves (and very often finds problems with them too). The problem is even worse on x86_64 with all of the multilib crap increasing the number of packages in the repos. However, I'm only actually interested in broken dependencies for packages in my own repositories, so most of the time spent by repoclosure is wasted for me.

My hack adds another option, --checkrepo, which works like the existing --repoid option, which specifies a repository that I'm interested in checking. Any other enabled repositories from the configuration file are used only to satisfy the dependencies and the packages there don't get their own dependencies checked. I'm seeing an order of magnitude improvement in checking times now, which is nice. Unfortunately my knowledge of python is very limited and hence it's a very horrible hack to get this working, and an unpleasant side effect is that I'm getting two copies of every logged message (i.e. every line of output) because I've now got two instances of the RepoClosure class rather than one. I can get rid of those using awk but I'd much rather find a cleaner solution. Must actually learn python someday.

Friday 8th June 2007

Local Packages

Fedora Project

Saturday 9th June 2007

Fedora Project

Allotment

Monday 11th June 2007

Local Packages

Fedora Upgrade

Made a start on upgrading my home machines to Fedora 7. First up was my firewall/web/DNS/ftp/mail server machine, which I've set up to be able to run two different OS releases, with the data shared between them. I only had time to do the base Fedora 7 install, and the only point of interest really was that I needed the libata.ignore_hpa=1 hack (see Bugs/F7Common) to get the partition table on the smaller of my 80GB disks to be recognised.

I wanted a way to choose between the two OS releases at boot time. With grub having been installed on the MBR, I only had the option of booting into the new Fedora 7 installation. As a quick fix, I copied the Fedora Core 6 boot entries from the Fedora Core 6 version of /etc/grub.conf and appended them to the Fedora 7 /etc/grub.conf file. That gave me a way to get into Fedora Core 6. A more long-term fix was to have each OS release's version of grub able to load the other OS's version.

I added the following to the Fedora Core 6 /etc/grub.conf:

title Fedora 7 Boot Menu
        rootnoverify (hd0,1)
        chainloader +1

and this to the Fedora 7 /etc/grub.conf

title Fedora Core 6 Boot Menu
        rootnoverify (hd0,0)
        chainloader +1

I have disks on this system installed as pairs and everything mirrored (RAID1). The Fedora Core 6 /boot is the first partition of my first hard disk pair, and the Fedora 7 /boot is the second partition. All that was needed now was to install each OS release's version of grub into the /boot partition for that OS release and then my two grub versions would be able to load each other.

I used the standard RAID1 grub installation route:

On Fedora Core 6:

# grub
grub>  device (hd0) /dev/hda
grub>  root (hd0,0)
grub>  setup (hd0,0)
grub>  device (hd0) /dev/hdb
grub>  root (hd0,0)
grub>  setup (hd0,0)
grub>  quit

On Fedora 7:

# grub
grub>  device (hd0) /dev/sda
grub>  root (hd0,1)
grub>  setup (hd0,1)
grub>  device (hd0) /dev/sdb
grub>  root (hd0,1)
grub>  setup (hd0,1)
grub>  quit

Note that in Fedora 7 all disks are regarded as being SCSI disks, so the device names change (check /boot/grub/device.map too).

Wednesday 13th June 2007

Fedora Upgrade

My firewall/web/DNS/mail/everything server is now running live with Fedora 7, and I have to say it was fairly painless to get it going, mainly thanks to the detailed install notes I kept when I did a fresh install of Fedora 6 many months ago.

Biggest issue was probably the need to update the SELinux policy for mod_fcgid (and hence this wiki), which was broken due to its use of deprecated permissions names. I've now fixed that, plus incorporated a boolean for allowing FastCGI applications to send mail, something I've been meaning to do for at least six months. I'll need to issue a Fedora 7 update for mod_fcgid containing this fix, which I'll look at doing tomorrow. A prerequsite for that is figuring out the version of selinux-polcy-devel that is now required to build the policy module; I suspect it won't build on FC5 and maybe not FC6 either.

Friday 15th June 2007

Local Packages

Fedora Project

Saturday 16th June 2007

Fedora Upgrade

Bit the bullet and upgraded my desktop/server/buildsystem machine from Fedora Core 6 to Fedora 7. The upgrade itself went OK until it was time to reboot. It wouldn't. I'd been bitten by Bug #242334, which resulted in my /etc/mdadm.conf having only one entry in it instead of the 6 it had originally. The result: no volume groups found at boot time. Having identified the problem, the fix was easy using the DVD in rescue mode, and the rest of the upgrade was relatively painless.

Monday 18th June 2007

Local Packages

I had a problem running repoview on Fedora 7:

ImportError: No module named elementtree.SimpleXMLWriter

This is Bug #232590. The quick workaround was to build and install the python-elementtree package from Fedora CVS, though it'll be removed courtesy of an obsolete as soon as a python update comes around. Hopefully there will be an update to repoview before then.

The recent pam update to version/release 0.99.7.1-5.1.fc7 caused failed dovecot logins due to SELinux denials:

type=AVC msg=audit(1182087900.637:32303): avc:  denied  { execute } for  pid=25443 comm="dovecot-auth" name="unix_update" dev=dm-0 ino=384037 scontext=user_u:system_r:dovecot_auth_t:s0 tcontext=system_u:object_r:updpwd_exec_t:s0 tclass=file

This was easily fixed in a local policy module and will apparently be fixed in selinux-policy 2.6.4-17 (Bug #244534).

Friday 22nd June 2007

Local Packages

Sunday 24th June 2007

Fedora Project

Monday 25th June 2007

Local Packages

Whilst building these packages, I noticed a worrisome problem in the buildsystem: mock was occasionally failing to download a package needed as a build requirement for something, but this was not preventing it from attempting to build the package. I noticed because the missing build requirements caused the actual package build to fail, but that might not always happen, and a package built with reduced functionality might be the result instead.

I raised this on fedora-buildsys-list and was asked for an easy way to reproduce the problem, which I managed to find. It remains to be seen whether it's a mock or a yum problem. Something to watch out for when doing builds though.

Tuesday 26th June 2007

Local Packages

I noticed a comment by MatthiasSaou in his changelog for proftpd:

* Fri Jun 15 2007 Matthias Saou <http://freshrpms.net/> 1.3.0a-4
- Fix PAM entry for F7+ (#244168). Still doesn't work with selinux, though.

Given that I use both proftpd and SELinux without problems, I wondered what this was about. Anyway, I tried doing a non-anonymous login (something I don't normally do as I use sftp for that sort of thing), and I saw these SELinux denials in the audit log:

type=AVC msg=audit(1182852462.003:103279): avc:  denied  { write } for pid=23852 comm="proftpd" scontext=root:system_r:ftpd_t:s0 tcontext=root:system_r:ftpd_t:s0 tclass=key
type=AVC msg=audit(1182852462.003:103279): avc:  denied  { link } for pid=23852 comm="proftpd" scontext=root:system_r:ftpd_t:s0 tcontext=root:system_r:dovecot_t:s0 tclass=key
type=AVC msg=audit(1182852462.003:103280): avc:  denied  { audit_control } for  pid=23852 comm="proftpd" capability=30 scontext=root:system_r:ftpd_t:s0 tcontext=root:system_r:ftpd_t:s0 tclass=capability

I raised this on fedora-selinux-list and the requisite policy changes were made in CVS. For the time being, I added the following in a local policy module:

# Allow ftpd to set the login uid and create a session keyring
logging_set_loginuid(ftpd_t)
logging_send_audit_msg(ftpd_t)
allow ftpd_t self:key { write search link };

I was then left with some remaining annoyances in /var/log/secure, such as:

Jun 26 12:09:42 goalkeeper proftpd: PAM audit_log_user_message() failed: Operation not permitted
Jun 26 12:09:42 goalkeeper proftpd[25559]: goalkeeper.intra.city-fan.org (::ffff:192.168.2.20[::ffff:192.168.2.20]) - PAM(setcred): System error
Jun 26 12:09:42 goalkeeper proftpd: pam_unix(proftpd:session): session closed for user paul
Jun 26 12:09:42 goalkeeper proftpd[25559]: goalkeeper.intra.city-fan.org (::ffff:192.168.2.20[::ffff:192.168.2.20]) - PAM(close_session): System error
Jun 26 12:09:42 goalkeeper proftpd[25559]: goalkeeper.intra.city-fan.org (::ffff:192.168.2.20[::ffff:192.168.2.20]) - FTP session closed.

To prevent these errors, proftpd needs to retain the CAP_AUDIT_WRITE capability when changing UID. It already has the facility to set capabilities (in mod_cap, enabled by default), but CAP_AUDIT_WRITE is not one of the capabilities handled by the module. I made a patch to support this and incorporated it into my local proftpd package (1.3.0a-5.2.CF). The result was the following messages for a session in /var/log/secure:

Jun 26 14:33:44 goalkeeper proftpd: pam_unix(proftpd:session): session opened for user paul by (uid=0)
Jun 26 14:33:44 goalkeeper proftpd[30169]: goalkeeper.intra.city-fan.org (::ffff:192.168.2.20[::ffff:192.168.2.20]) - USER paul: Login successful.
Jun 26 14:33:48 goalkeeper proftpd: pam_unix(proftpd:session): session closed for user paul
Jun 26 14:33:48 goalkeeper proftpd[30169]: goalkeeper.intra.city-fan.org (::ffff:192.168.2.20[::ffff:192.168.2.20]) - FTP session closed.

Annoying error messages gone :-)

I passed the patch on to Matthias to see what he thinks; if all seems well, I'll forward it upstream too.

Wednesday 27th June 2007

Wiki

Developed and wrote SubsetRepositories.

Thursday 28th June 2007

Local Packages

Friday 29th June 2007

Local Packages

Previous Month: May 2007
Next Month: July 2007