Access Encrypted ZFS from Rescue Boot
How I Lost Control
When working with
root rights on Linux, you’ll always find a possibility to shoot yourself in the foot. My choice for self-harm this time was to use
usermod in a wrong way. The whole story turned worse, because I was on my very new Laptop, that I bough yesterday: As always, purged Windoze and installed Ubuntu 22.04. As always: Works like a charm! But I thought it was a good idea, to install the filesystem on encrypted ZFS, instead of an encrypted LVM, as I always did before. But ZFS has been reported to be cool by some geeky friends, so who needs to know what he does, when you just can click a checkbox in a UI.
docker, you should add yourself to the group
docker to be able to use docker without
sudo. A normal person would read the best blog entry on docker to get the exact command. But not me, I thought it’s a good idea to write the command from my mind:
usermod -G docker marc adds group
docker to user
marc — I thought… Well, lazyness pays off often, but not always. This time, I had better read the man page before:
-G, --groups GROUP1[,GROUP2,...[,GROUPN]]] A list of supplementary groups which the user is also a member of. Each group is separated from the next by a comma, with no intervening whitespace. The groups are subject to the same restrictions as the group given with the -g option. If the user is currently a member of a group which is not listed, the user will be removed from the group. This behaviour can be changed via the -a option, which appends the user to the current supplementary group list.
So a simple
usermod -aG docker marc would have done the trick. But without the little
docker was not appended to my groups, but it replaced all my groups, so I was thrown out of all other groups, including the
admin and many more. With that, I lost all administration rights on my laptop;
sudo was no more possible. Of course, I noted that today, one day later, because it was effective only after a reboot. Would I have noticed on time, I still could have fixed it.
But now, I had wrong entries in
/etc/group and couldn’t change them without
Fix Broken Root Access on Linux
To fix lost root access on Linux, simply boot e.g. a Ubuntu Desktop as rescue system from an USB stick. There you can install or try Ubuntu, just try Ubuntu, then you’re in a full Ubuntu environment, where you have access. Just mount the laptop disk to fix the problem.
And because that’s so easy (not only on Linux, but on every system), you must always encrypt your installations!
Mount Encrypted ZFS on Ubuntu 22.04
Ok, mounting an encrypted LVM is hard enough, but at least you can google it, and you find the necessary instructions. But decrypting a ZFS from a rescue system? Finally, I found this: Rescuing using a Live CD. Unfortunately, the very first line is bullshit.
Ignore that for now and start by initializing the ZFS pools:
sudo zpool export -a sudo zpool import -N -R /mnt rpool sudo zpool import -N -R /mnt bpool
After searching around and analyzing my Ubuntu installation, I found, that ZFS is not encrypted directly by the harddisk encryption password you set during the installation, but your key encrypts a LUKS block device, this partition contains a file, that file contains the real decryption key. This block device is
/dev/zvol/rpool/keystore, which is a link to
/dev/zd0 and decrypted device is named by convention
keystore-rpool. So that’s what you need to decrypt with your password:
sudo cryptsetup luksOpen /dev/zd0 keystore-rpool
By convention, this has to be mounted to
/run/keystore/rpool, so that it is found and accessed by
zfs load-key -a, which finally decrypts the ZFS. That’s all, so mount, decrypt and clean up:
sudo mkdir -p /run/keystore/rpool sudo mount /dev/mapper/keystore-rpool /run/keystore/rpool sudo zfs load-key -a sudo umount /run/keystore/rpool sudo cryptsetup luksClose keystore-rpool
Now you need to find your ZFS’
zfs list shows it, but in a large output, so get that cleaner:
zfs list | sed -n 's/^\(rpool\/ROOT\/ubuntu_[^/ ]*\).*/\1/p' | uniq
This outputs something like:
ubuntu_iukzaq, instead of:
sudo zfs mount rpool/ROOT/ubuntu_UUID sudo zfs mount bpool/BOOT/ubuntu_UUID sudo zfs mount -a
ubuntu_iukzaq and mount ZFS typing:
sudo zfs mount rpool/ROOT/ubuntu_iukzaq sudo zfs mount bpool/BOOT/ubuntu_iukzaq sudo zfs mount -a
Now my Ubuntu root filesystem is mounted at
/mnt and I can repair my
/etc/group file in
How To Decrypt a Filesystem Without Password
Not. That’s the idea of encryption.
If you forgot your password, or if you crash your key file, your data is lost. So take care of your passwords and keyfiles, and do backups of your valuable data!