Making pam_mount play nice(r) with systemd user sessions

I like pam_mount‘s idea of providing per-user, on-demand, block device level encryption. Sadly, it turns out it doesn’t really play nicely with systemd user sessions, which are enabled by default in Arch Linux (you can disable them, but it screws up stuff like PulseAudio socket activation). The Arch wiki warns upfront that pam_mount can also unmount your partitions when you close your last session but this is currently not working due to the use of pam_systemd.so in the PAM stack. While pam_mount and systemd indeed have some disagreements, they can be persuaded to play together quite nicely with some fiddling.

Assume you want a user to have a LUKS-encrypted volume mounted at $HOME upon each login and automatically unmounted when that user has no more active sessions on the host. First, let’s try what the Arch wiki says. Install pam_mount, then add something like this to /etc/security/pam_mount.conf.xml, before the closing </pam_mount> tag:

<lclmount>mount -t%(FSTYPE) %(VOLUME) %(MNTPT) "%(if %(OPTIONS),-o%(OPTIONS))"</lclmount>
<volume user="kempniu" fstype="auto" path="/dev/mapper/luks-kempniu" mountpoint="/home/kempniu" options="fsck,noatime" />

Then, add pam_mount.so to /etc/pam.d/system-auth:

auth      required  pam_unix.so     try_first_pass nullok
auth      optional  pam_mount.so
...
session   optional  pam_mount.so
session   required  pam_limits.so

Next, try to log in. The process succeeds, but takes quite a while, more than you would expect from a simple mount invocation. Taking a peek into the logs reveals error messages:

Jan 07 09:31:40 systemd[744]: (pam_mount.c:173): conv->conv(...): Conversation error
Jan 07 09:31:40 systemd[744]: (pam_mount.c:477): warning: could not obtain password interactively either
Jan 07 09:31:40 systemd[744]: (mount.c:68): Messages from underlying mount program:
Jan 07 09:31:40 systemd[744]: (mount.c:72): crypt_activate_by_passphrase: File exists
Jan 07 09:31:45 systemd[744]: (pam_mount.c:522): mount of /dev/mapper/luks-kempniu failed
Jan 07 09:31:45 systemd[744]: (pam_mount.c:173): conv->conv(...): Conversation error
Jan 07 09:31:45 systemd[744]: (pam_mount.c:477): warning: could not obtain password interactively either
Jan 07 09:31:45 systemd[744]: pam_unix(systemd-user:session): session opened for user kempniu by (uid=0)

Note the 5 second gap in the middle. That’s exactly what happens when you try to mount an already mounted LUKS volume:

# time mount /dev/mapper/luks-kempniu /home/kempniu </dev/null
Password: 
crypt_activate_by_passphrase: File exists

real    0m4.812s
user    0m0.003s
sys     0m0.000s

So it seems pam_mount is needlessly called twice, causing a delay. This is consistent with the contents of /var/run/pam_mount/kempniu (the file holding the current session count for the user) – even though we only logged in once, the session counter is at 2. This happens due to pam_systemd.so being present in /etc/pam.d/system-login – the first time each user logs in, systemd-logind launches an instance of systemd --user for that user, which causes PAM authentication to happen again (because /etc/pam.d/systemd-user simply includes /etc/pam.d/system-login). As the user’s volume is already decrypted and mounted at this point, there is no need to invoke pam_mount again. Let’s simply prevent pam_mount.so from being used when authentication is performed for systemd --user by adding a relevant pam_succeed_if.so entry to /etc/pam.d/system-auth:

session   [success=1 default=ignore]  pam_succeed_if.so  service = systemd-user quiet
session   optional  pam_mount.so
session   required  pam_limits.so

Things are looking way better now – login time has decreased, no error messages appear in the logs after logging in and the volume is unmounted upon logout (as the user session count at /var/run/pam_mount/kempniu is now correct). Yet, if we try to log in again while the user’s first session is still active, it takes a few seconds to get to the shell prompt, even though the user’s volume is already decrypted and mounted. Also, error messages start appearing again:

Jan 07 09:54:44 sshd[476]: (mount.c:68): Messages from underlying mount program:
Jan 07 09:54:44 sshd[476]: (mount.c:72): crypt_activate_by_passphrase: File exists
Jan 07 09:54:49 sshd[476]: (pam_mount.c:522): mount of /dev/mapper/luks-kempniu failed

This is all expected – even though pam_mount is now only invoked once per login, it still just foolishly assumes the volume is not yet mounted and calls mount anyway, causing a delay.

To fix that, let’s write a simple mounting script which makes sure that the user’s volume (supplied as the first command line argument) is not yet mounted and only then calls the actual mount command:

#!/bin/sh

if cut -d' ' -f2 /proc/self/mounts | grep -F -x -q -- "$1"; then
    exit 0
fi

shift
exec mount "$@"

Now, let’s hook the script up to pam_mount in /etc/security/pam_mount.conf.xml (note the extra %(MNTPT) token):

<lclmount>/usr/local/bin/mount.ifnotmounted %(MNTPT) -t%(FSTYPE) %(VOLUME) %(MNTPT) "%(if (%OPTIONS),-o%(OPTIONS))"</lclmount>

Simple command-line tests might create an illusion that this solves all remaining problems: error messages don’t appear any more¹, any further logins after the initial one are immediate and the volume is correctly unmounted after the user’s last session ends. However, things will still break if the systemd user instance starts any process (like PulseAudio), because PAM modules are always processed in the same order, no matter whether the session is opened or closed. We have to put pam_mount.so before pam_systemd.so in the stack, because otherwise systemd --user won’t be able to read its configuration from a yet-to-be-decrypted volume (~/.config/systemd/user). Doing this, however, prevents pam_mount from successfully unmounting the user’s volume after systemd --user spawns any child process:

Jan 07 10:22:17 sshd[1140]: (mount.c:68): umount messages:
Jan 07 10:22:17 sshd[1140]: (mount.c:72): umount: /home/kempniu: target is busy
Jan 07 10:22:17 sshd[1140]: (mount.c:72):         (In some cases useful info about processes that
Jan 07 10:22:17 sshd[1140]: (mount.c:72):          use the device is found by lsof(8) or fuser(1).)
Jan 07 10:22:17 sshd[1140]: (mount.c:72): umount /home/kempniu failed with run_sync status 2

To work around this, we’ll tell pam_mount to unmount volumes using yet another custom script (taking username and mount point as command line arguments), which will first shut down the systemd user instance, then wait for a while to give the processes spawned by it a chance to exit cleanly and finally call umount:

#!/bin/sh

XDG_RUNTIME_DIR="/run/user/`id -u "$1"`" su "$1" -c "systemctl --user exit"

TIMEOUT=3

while [ ${TIMEOUT} -gt 0 ]; do
    if ! fuser -m "$2" >/dev/null 2>&1; then
        exec umount "$2"
    fi
    sleep 1
    TIMEOUT=$((${TIMEOUT}-1))
done

While this is clearly a hack, systemd-logind doesn’t seem to mind. To hook the script up to pam_mount, add the following line to /etc/security/pam_mount.conf.xml, just below the <lclmount> line:

<umount>/usr/local/bin/umount.stopsystemd %(USER) %(MNTPT)</umount>

I’ve been using such a contraption for quite a while now and it’s been working surprisingly well, considering the scripts’ naïveness. So far I have only found one downside – if you leave a process running in background which uses (e.g. as its current working directory) the user’s home directory upon logout, your systemd user instance will be shutdown anyhow and the unmount will obviously fail. The simplest solution is simply not to start background processes from your home directory (duh!). A real one, though, would be to make the unmounting script a little bit smarter, which is left as an excercise for the reader ;)

---

1. In some cases, like when using OpenSSH with public key authentication, upon second and further simultaneous logins you’ll still get some warnings that are safe to ignore (conv->conv(…): Conversation error, warning: could not obtain password interactively either). They just mean that pam_mount was not able to fetch the password from the PAM stack, which is expected, because no password was entered by the user at all, but we don’t really care because the LUKS volume is already decrypted and mounted. ↩