so i’ve got my gpg smartcard compatible hardware key now, was thinking about using it to hold my filesystem encryption keys. and i went around and didn’t find anything about openzfs supporting any of the non-trivial standards, pkcs or anything else i’ve yet looked into. there’s one folk posting on reddit a bunch of a apparently dead links claiming to be a “zfs 2fa encryption helper for yubikey”. well i guess even if the link isn’t dead, i would still want to figure out how it work before i apply any of those to my own data.

i started to look into my own solution because how hard can it be, like i’m pretty sure i can just somehow copy the procedure in luks and make it work for zfs instead. the original plan is to generate a key (in raw format) for this zfs dataset, encrypt it with gpg (which yubikey and mine obviously support), put it into my boot partition, and configure initrd to decrypt and load that key for me; i’d just need to enter the pin protecting my key, and touch the button on it, and system does the rest of stuff for me, without ever exposing the private key, and hopefully keep the zfs symmetric keys in memory.

echo "meowmeow" | gpg --encrypt --recipient hexadecimaaal@gmail.com
# 0�b�z�U%�I�%���,�0�'�����s���D�[����*�)f}�Y媠���
#''�J�.=�����)0/��A������ɓ-X��"$A��F��;���{g!�MZ�&��y��{��
#                                                         ��e��a����?
# you get the idea

echo "meowmeow" | gpg --encrypt --recipient hexadecimaaal@gmail.com | gpg --decrypt
# Please insert the card with serial number:
# 
#   F1D0 [redacted]
#   OK
#   Cancel
# [oc]? o
# Please unlock the card
# 
# Number: F1D0 [redacted]
# Holder: Hexadecimal 
# PIN: 
# gpg: encrypted with 255-bit ECDH key, ID [redacted], created [redacted]
#       "Hexadecimal <hexadecimaaal@gmail.com>"
# meowmeow

i thought the best way is to keep it all in a chain of standard unix piping, and just let the shell automatically destroy those pipes for me, that would be less of a hassle to track down and clean those up after the key is safely guarded to kernel space.

echo "meowmeow" | gpg --encrypt --recipient hexadecimaaal@gmail.com --armor
# -----BEGIN PGP MESSAGE-----
# 
# hF4DRLkUNiQAMo4SAQdAoe/dhWTcm4JFDW2H21Z/abH30Vf/SmMsRMSQ4l7EuQAw
# SoUmW4glrJWWfYn/4rll9+t/h9U9IkTlqF63bzaCxV8MpBDCdjmaSBbjlL0VjqJI
# 0kEB0pfqu6xBlA82Fz2UwE0GDoFuYHTcJO65eLSYQE6Ty0xtz8cKtIObIl5am5B3
# oFvTOiPTCSbhc7gbJf+vybPvmQ==
# =0n9U
# -----END PGP MESSAGE-----

after i played with gpg cmdline for a while, i feel like the default output mode makes more sense for a script, than the “armor” output that emits the base64 encoded version and a bunch of fixed header text. but then it’s rather hard to work with in terminal, and i need to play with it to make sure i understand how everything work… and i suddenly remembered, that i can set and get zfs dataset property fairly easily on cmdline, and the thing is, they are available before i load the encryption key for the dataset. and even better is that i don’t have to worry about blasting my efi partition someday for whatever reason, and then remembered i haven’t backed up the key yet… so that’s the right way to go, zfs properties.

echo "meowmeow" | gpg --encrypt --recipient hexadecimaaal@gmail.com | base64 -w0
# hF4DRLkUNiQAMo4SAQdAJ3oKnR3kEs2N4j740OggvsKKM4pn0hs5mIAlVZ3mzgYwFF1UFGPW5+iJppQ/rW0WHPrLbf0xA5i6XCceTG/DLHhhNVvW5MRYrIUzZjg8wIu/0kEBABqTYjfxATFuUP5InlsE0qNJVxQS43/Kb8KSq8Ub/rmcpHu2xUbcaibYm7gMxias2QU+LyDnEGIbueGm3ZkyKw==

echo "meowmeow" | gpg --encrypt --recipient hexadecimaaal@gmail.com | base64 -w0 | base64 -d | gpg --decrypt
# gpg: encrypted with 255-bit ECDH key, ID [redacted], created [redacted]
#      "Hexadecimal <hexadecimaaal@gmail.com>"
# meowmeow

the next thing is, i’m pretty sure zfs won’t let me send the raw binary data through its zfs set command, that this command only accepts data as argument written directly on cmdline. so, base64 it then.

sudo zfs set ca.hexade:gpg-cipher=hF4DRLkUNiQAMo4SAQdAJ3oKnR3kEs2N4j740OggvsKKM4pn0hs5mIAlVZ3mzgYwFF1UFGPW5+iJppQ/rW0WHPrLbf0xA5i6XCceTG/DLHhhNVvW5MRYrIUzZjg8wIu/0kEBABqTYjfxATFuUP5InlsE0qNJVxQS43/Kb8KSq8Ub/rmcpHu2xUbcaibYm7gMxias2QU+LyDnEGIbueGm3ZkyKw== salt/test
# for testing (and demonstration) purpose, i just copied the last output and 
# pasted it here. i'll go over how to automate this (at least for a little bit).

zfs get ca.hexade:gpg-cipher salt/test
# NAME       PROPERTY              VALUE                                                                                                                                                                                                                         SOURCE
# salt/test  ca.hexade:gpg-cipher  hF4DRLkUNiQAMo4SAQdAJ3oKnR3kEs2N4j740OggvsKKM4pn0hs5mIAlVZ3mzgYwFF1UFGPW5+iJppQ/rW0WHPrLbf0xA5i6XCceTG/DLHhhNVvW5MRYrIUzZjg8wIu/0kEBABqTYjfxATFuUP5InlsE0qNJVxQS43/Kb8KSq8Ub/rmcpHu2xUbcaibYm7gMxias2QU+LyDnEGIbueGm3ZkyKw==  local

# to make the output more workable, supply zfs-get with `-H` (omit header) and
# `-o value` (well, only output column "value").

zfs get -H -o value ca.hexade:gpg-cipher salt/test
# hF4DRLkUNiQAMo4SAQdAJ3oKnR3kEs2N4j740OggvsKKM4pn0hs5mIAlVZ3mzgYwFF1UFGPW5+iJppQ/rW0WHPrLbf0xA5i6XCceTG/DLHhhNVvW5MRYrIUzZjg8wIu/0kEBABqTYjfxATFuUP5InlsE0qNJVxQS43/Kb8KSq8Ub/rmcpHu2xUbcaibYm7gMxias2QU+LyDnEGIbueGm3ZkyKw==

and i was thinking about finding some random attribute to jug my encrypted raw key in, like some field for commentary or something, notes? but sadly to no avail. then yet again i remembered there’s this “user zfs property” thing, basically lets you write whatever you like into a zfs dataset metadata section. after poking around and googling fanatically for a while, the correct format for user property name turns out to be, upper case letters aren’t allowed, and the name must include at least one semicolon. so i decided to call this field ca.hexade:gpg-cipher, you can name it whatever you like in your version.

# (for testing purpose, unload-key first.)
zfs get -H -o value ca.hexade:gpg-cipher salt/test | base64 -d | gpg --decrypt | sudo zfs load-key salt/test
# gpg: encrypted with 255-bit ECDH key, ID [redacted], created [redacted]
#      "Hexadecimal <hexadecimaaal@gmail.com>"

so it works now, great. next thing down the line is to make sure it can unlock with other GNUPG_HOME set, since we’ll only have initramfs loaded, we have to unlock it with a completely bare GNUPG_HOME. and again, after intense poking around, i found out that, gpg just don’t want to use your private key (or secret-key as in gpg terminology) if you don’t have the corresponding public key in GNUPG_HOME. the standard way of coming around this (with your gpg smartcard compatibles) is to store an url in your card for downloading your gpg public key. we’ll need it in initrd, which probably won’t have access to your network interfaces, let alone connecting to the internet1. so i figured the best way is probably just store the public key besides the cipher.

gpg --export [your keygrip] | base64 -w0 | wc -c
# 1184

and i’m kind of afraid this is too long for a zfs property. looked through something written by oracle, the zfs in solaris (which is very diverged from openzfs now) says a user property can have 1024 characters in them at most, and can have a name 256 characters long at most. i figured to just give it a spin and see what comes, and it didn’t complain about anything. i stored this in an attribute i decided to call ca.hexade:gpg-pubkey.

zfs get -H -o value ca.hexade:gpg-pubkey salt/test | base64 -d | gpg --import
# ... gpg: Total number processed: 1 ...

now the setup is good to go, we just need to tell initrd how to do these things automatically. again again, after intense bash-fu poking and consulting with dram and re-stackoverflow-ing, here’s my version of initrd script for the process:

gpg-agent --daemon \
    --scdaemon-program /bin/scdaemon \
    --allow-loopback-pinentry

zfs list -H -o name | while IFS= read -r dataset; do
pubkey="$(zfs get -H -o value ca.hexade:gpg-pubkey "$dataset")"
if [ "$pubkey" != "-" ]; then
    echo "$pubkey" | base64 -d | gpg --import
fi
done

gpg --card-status > /dev/null 2> /dev/null

while ! gpg --card-status > /dev/null 2> /dev/null; do
read -p "GPG smartcard not present. try again? (Y/n)" input
if [ "$input" == "n" -o "$input" == "N" ]; then
    break
fi
done

exec 13< <(zfs list -H -o name)
# we need another fifo here because there's another `read` in the loop later
while IFS= read -r dataset <&13; do
cipher="$(zfs get -H -o value ca.hexade:gpg-cipher "$dataset")"
if [ "$cipher" != "-" ]; then # zfs returns `-` for valid but unavailable properties
    for i in $(seq 3); do 
    # sadly, none of the `pinentry-program`s work in initrd. 
    # so we have to do it ourselves.
    read -s -p "enter GPG smartcard PIN:" pin
    echo "$cipher" | base64 -d \
    | gpg --batch --decrypt --pinentry-mode loopback \
        --passphrase-file <(echo "$pin") \
    | zfs load-key "$dataset"
    if [ "$?" == 0 ]; then break; fi
    done
fi
done
exec 13<&-

zfs load-key -a

might be a better way out there to cache your smartcard pin and use it later, but i have only one encryptionroot anyways, and i think my version works for multiple hardware keys. don’t want to introduce more complexity. use this to replace whatever section responsible for unlocking your zfs datasets in initrd. and also remember to change the script responsible of composing your initramfs, copy gpg gpg-agent scdaemon and their libraries to initramfs archive. takeaway for nixos:

boot.zfs.requestEncryptionCredentials = [];

boot.initrd = {
    extraUtilsCommands = ''
        copy_bin_and_libs ${pkgs.gnupg}/bin/gpg
        copy_bin_and_libs ${pkgs.gnupg}/bin/gpg-agent
        copy_bin_and_libs ${pkgs.gnupg}/libexec/scdaemon
    '';

    extraUtilsCommandsTest = ''
        $out/bin/gpg --version
        $out/bin/gpg-agent --version
        $out/bin/scdaemon --version
    '';

    postDeviceCommands = lib.mkAfter ''
        gpg-agent --daemon \
        --scdaemon-program ${pkgs.gnupg}/libexec/scdaemon \
        --allow-loopback-pinentry

        zfs list -H -o name | while IFS= read -r dataset; do
        pubkey="$(zfs get -H -o value ca.hexade:gpg-pubkey "$dataset")"
        if [ "$pubkey" != "-" ]; then
            echo "$pubkey" | base64 -d | gpg --import
        fi
        done

        gpg --card-status > /dev/null 2> /dev/null

        while ! gpg --card-status > /dev/null 2> /dev/null; do
        read -p "GPG smartcard not present. try again? (Y/n)" input
        if [ "$input" == "n" -o "$input" == "N" ]; then
            break
        fi
        done

        exec 13< <(zfs list -H -o name)
        while IFS= read -r dataset <&13; do
        cipher="$(zfs get -H -o value ca.hexade:gpg-cipher "$dataset")"
        if [ "$cipher" != "-" ]; then
            for i in $(seq 3); do
            read -s -p "enter GPG smartcard PIN:" pin
            echo "$cipher" | base64 -d \
            | gpg --batch --decrypt --pinentry-mode loopback \
                --passphrase-file <(echo "$pin") \
            | zfs load-key "$dataset"
            if [ "$?" == 0 ]; then break; fi
            done
        fi
        done
        exec 13<&-

        zfs load-key -a
    '';
};

PS: was thinking about using some automatic tool to capitalize text for me after finished writing, but doesn’t seem to be that easy. anyways.

  1. well many people actually set up their device to ssh to somewhere and load key from there, i don’t want to jump through these hoops and i want to use it on my future laptop, which i guess won’t have connection to said ssh server anyways. ↩