Converting a Pop_OS! 21.10 installation to BTRFS

Diego Zaccariotto

Last updated on Apr 3, 2022 9 min read Linux

Smart

Steps to convert Pop_OS! 21.10 standard installation to btrfs

What is this guide about

I wrote this guide as a note to self as I did originally install my laptop using the standard procedure of Pop_OS! that uses ext4 and I wanted to move to btrfs without having to reinstall everything.

This guide is HEAVILY based on “DRAFT: Pop!_OS 21.10: installation guide with btrfs-LVM-luks and auto snapshots with BTRBK” from Willi Mutschler, available at https://mutschler.dev/linux/pop-os-btrfs-21-10/ (accessed on 2022-03-30).

When I say heavily based I mean that this basically is Mutschler’s guide. I did a number of tests on VMs and a couple of live systems and documented the additional steps needed for the conversion. If it wasn’t for Mutschler’s documentation it would have taken me way, way longer to prepare this guide.

Make sure you do have a backup of your data before making any change to your filesystem. Data loss is a concrete opportunity here.

Why BTRFS?

In my case the decision mainly came from the ability to manage snapshots and the great integration with timeshift and timeshift-autosnap-apt (https://github.com/wmutschl/timeshift-autosnap-apt ).

If you are interested in more details about BTRFS you can check out the official website: btrfs Wiki

Assumptions

This procedures assumes that Pop_OS! was installed using the default installation w/ encryption (efi partition (fat32), recovery partition (fat32), luks+lvm+ext4) for /, swap). I used this procedure successfully also on machines where the swap partition was moved within the encrypted luks volume, which is something you do when following the official guide from System76 to enable hibernation on Pop_OS! (Enable Hibernation (Suspend to Disk) - System76 Support)

Here I’m assuming your disk is /dev/sda. Just run lsblk or fdisk -l to check your partitions if you are not sure what your disk is.

edit /boot/efi/loader/loader.conf (you need to be root or use sudo)

add timeout 2 at the end of file so to be able to easily boot into the recovery partition. Your ending file should look like the following:

default Pop_OS-current
timeout 2

Install the btrfs-progs package as Pop_OS! does not install it by default:

sudo apt install -y btrfs-progs

(mind that this will re-generate the initramfs images and it takes some time). You might want to download the package file on the local filesystem as well; this will allow you not to have to download the package from the recovery partition (see in two steps).

reboot into the recovery partition

open a terminal and run

sudo -i

to enter interactive mode (root)

open the luks volume:

cryptsetup luksOpen /dev/sda3 cryptdata

VERY IMPORTANT: you need to force a check on the original ext4 filesystem. You do not want to try and convert an inconsistent filesystem!!!

Running the conversion on a inconsistent filesystem can easily lead to data loss.

Run:

fsck.ext4 -f /dev/mapper/data-root

download btrfs-progs either from pkgs.org or just do a web search, use amd64 as architecture. If you downloaded the package in the previous steps you can just mount your filestystem and install from there. At the time of writing of this guide the package you are looking for is: btrfs-progs_5.10.1-2build1_amd64.deb.

Install btrfs-progs using dpkg:

dpkg -i btrfs-progs_5.10.1-2build1_amd64.deb

(it’s not installed by default in the main distribution, it’s not installed in the recovery partition either…)

if you did mount your data-root partition as you had your btrfs-progs there, unmount it:

cd /; umount /mnt

assuming you mounted it on /mnt.

convert / from ext4 to btrfs:

btrfs-convert /dev/mapper/data-root

The conversion can take quite same time. When I run it on my laptop it took about 50 minutes for a filesystem with 230 Gb in use on an SSD, intel i5, 16Gb of ram. Be patient.

Mount the newly converted volume (the ssd and discard options are specifically for SSDs or NVME disks):

mount -o subvolid=5,ssd,noatime,space_cache,commit=120,compress=zstd,discard=async /dev/mapper/data-root /mnt

create the subvolume for / and /home as @ and @home

btrfs subvolume create /mnt/@
# Create subvolume '/mnt/@'
cd /mnt
ls | grep -v @ | xargs mv -t @ #move all files and folders to /mnt/@
ls -a /mnt
# . .. @

btrfs subvolume create /mnt/@home

btrfs subvolume create /mnt/@home
# Create subvolume '/mnt/@home'
mv /mnt/@/home/* /mnt/@home/
ls -a /mnt/@/home
# . ..
ls -a /mnt/@home
# . .. diego

btrfs subvolume list /mnt

Changes to fstab

We need to adapt the fstab to

mount / to the @ subvolume
mount /home to the @home subvolume
make use of optimized btrfs mount options

So open it with a text editor, e.g.:

nano /mnt/@/etc/fstab

blkid -s UUID -o value /dev/mapper/data-root

UUID=(id_from_the_above) / btrfs  defaults,subvol=@,ssd,noatime,space_cache,commit=120,compress=zstd,discard=async 0 0
UUID=(id_from_the_above) /home btrfs  defaults,subvol=@home,ssd,noatime,space_cache,commit=120,compress=zstd,discard=async 0 0

Either way your fstab should look like this:

cat /mnt/@/etc/fstab
# PARTUUID=6b533522-0c33-4f44-890f-4be275c5b06f  /boot/efi  vfat  umask=0077  0  0
# PARTUUID=45bb9da4-9571-40bc-8f20-468332234a62  /recovery  vfat  umask=0077  0  0
# /dev/mapper/cryptswap  none  swap  defaults  0  0
# UUID=591dae2e-37ce-42c9-8ceb-5b124658ca6a  /  btrfs  defaults,subvol=@,ssd,noatime,space_cache,commit=120,compress=zstd,discard=async  0  0
# UUID=591dae2e-37ce-42c9-8ceb-5b124658ca6a  /home  btrfs  defaults,subvol=@home,ssd,noatime,space_cache,commit=120,compress=zstd,discard=async   0 0

Note that your PARTUUID and UUID numbers will be different. The last two lines for / and /home are the important ones.

Changes to crypttab

As we use discard=async, we need to add discard to the crypttab:

sed -i 's/luks/luks,discard/' /mnt/@/etc/crypttab
cat /mnt/@/etc/crypttab
# cryptdata UUID=c5b8099a-f035-47fb-939f-fa4ea770a403 none luks,discard
# cryptswap UUID=52de8233-c50b-4873-b586-9ab313d28b56 /dev/urandom swap,plain,offset=1024,cipher=aes-xts-plain64,size=512

Adjust configuration of kernelstub

We need to adjust some settings for the systemd boot manager and also make sure these settings are not overwritten if we install or update kernels and modules. Namely, we need to add rootflags=subvol=@ to the "user" kernel options of the kernelstub configuration file:

nano /mnt/@/etc/kernelstub/configuration

Here you need to add rootflags=subvol=@ to the "user" kernel options. That is, your configuration file should look like this:

cat /mnt/@/etc/kernelstub/configuration
# {
#   "default": {
#     "kernel_options": [
#       "quiet",
#       "splash"
#     ],
#     "esp_path": "/boot/efi",
#     "setup_loader": false,
#     "manage_mode": false,
#     "force_update": false,
#     "live_mode": false,
#     "config_rev":3
#   },
#   "user": {
#     "kernel_options": [
#       "quiet",
#       "loglevel=0",
#       "systemd.show_status=false",
#       "splash",
#       "rootflags=subvol=@"
#     ],
#     "esp_path": "/boot/efi",
#     "setup_loader": true,
#     "manage_mode": true,
#     "force_update": false,
#     "live_mode": false,
#     "config_rev":3
#   }
# }

Don’t forget the comma after "splash" (in the line above your added "rootflags=subvol=@" option) , otherwise you get errors when you later run update-initramfs (see below)!

Adjust configuration of systemd bootloader

We need to adjust some settings for the systemd boot manager, so let’s mount our EFI partition

mount /dev/sda1 /mnt/@/boot/efi

Add rootflags=subvol=@ to last line of Pop_OS_current.conf either using a text editor or the following command

sed -i 's/splash/splash rootflags=subvol=@/' /mnt/@/boot/efi/loader/entries/Pop_OS-current.conf
cat /mnt/@/boot/efi/loader/entries/Pop_OS-current.conf
# title Pop!_OS
# linux /EFI/Pop_OS-UUID_of_data-root/vmlinuz.efi
# initrd /EFI/Pop_OS-UUID_of_data-root/initrd.img
# options root=UUID=UUID_of_data-root ro quiet loglevel=0 systemd.show_status=false splash rootflags=subvol=@

where UUID_of_data-root is the UUID of /dev/mapper/data-root (in the previous steps you have the syntax for blkid to get it, but you can just take it from fstab at this point)

Create a chroot environment and update initramfs

Now, let’s create a chroot environment, which enables you to work directly inside your newly installed OS, without actually rebooting. For this, unmount the top-level root filesystem from /mnt and remount the subvolume @ which we created for / to /mnt:

cd /
umount -l /mnt
mount -o defaults,subvol=@,ssd,noatime,space_cache,commit=120,compress=zstd,discard=async /dev/mapper/data-root /mnt

Then the following commands will put us into our system using chroot:

for i in /dev /dev/pts /proc /sys /run; do mount -B $i /mnt$i; done
chroot /mnt

You are now inside your system and we can check whether our fstab mounts everything correctly:

mount -av
# /boot/efi                : successfully mounted
# /recovery                : successfully mounted
# none                     : ignored
# /                        : ignored
# /home                    : successfully mounted

Looks good! Now we need to update the initramfs to make it aware of our changes:

update-initramfs -c -k all

Note that if you run into errors like this:

update-initramfs: Generating /boot/initrd.img-5.11.0-7620-generic
kernelstub.Config    : INFO     Looking for configuration...
Traceback (most recent call last):
  File "/usr/bin/kernelstub", line 244, in <module>
    main()
  File "/usr/bin/kernelstub", line 241, in main
    kernelstub.main(args)
  File "/usr/lib/python3/dist-packages/kernelstub/application.py", line 142, in main
    config = Config.Config()
  File "/usr/lib/python3/dist-packages/kernelstub/config.py", line 50, in __init__
    self.config = self.load_config()
  File "/usr/lib/python3/dist-packages/kernelstub/config.py", line 60, in load_config
    self.config = json.load(config_file)
  File "/usr/lib/python3.9/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.9/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 20 column 7 (char 363)
run-parts: /etc/initramfs/post-update.d//zz-kernelstub exited with return code 1

you probably forgot a comma after "splash" in the /etc/kernelstub/configuration file (see above).

Reboot, some checks, and btrfs optimization

Now, it is time to exit the chroot and reboot the system:

exit

reboot now

If all went well you should see a single passphrase prompt, where you enter the luks passphrase and your system boots.

Now let’s click through the welcome screen and open a terminal to see whether everything is set up correctly (if you reached this point you have done the hard part):

## check that everything is mounted correctly
sudo mount -av
# /boot/efi                : already mounted
# /recovery                : already mounted
# none                     : ignored
# /                        : ignored
# /home                    : already mounted

## check that root and /home are correctly mounted
sudo mount -v | grep /dev/mapper
# /dev/mapper/data-root on / type btrfs (rw,noatime,compress=zstd:3,ssd,discard=async,space_cache,commit=120,subvolid=265,subvol=/@)
# /dev/mapper/data-root on /home type btrfs (rw,noatime,compress=zstd:3,ssd,discard=async,space_cache,commit=120,subvolid=266,subvol=/@home)

##  check the swap partition
sudo swapon
# NAME      TYPE      SIZE USED PRIO
# /dev/dm-2 partition   4G   0B   -2

## show the btrfs filesystem on /
sudo btrfs filesystem show /
# Label: none  uuid: 591dae2e-37ce-42c9-8ceb-5b124658ca6a
#     Total devices 1 FS bytes used 8.15GiB
#     devid    1 size 468.43GiB used 10.02GiB path /dev/mapper/data-root

## check what subvolumes exist on  /
sudo btrfs subvolume list /
# ID 264 gen 82 top level 5 path ext2_saved
# ID 265 gen 82 top level 5 path @
# ID 266 gen 82 top level 5 path @home

The checks performed so far as the some as seen on Mutscher’s page, the steps below are taken from the man-page of btrfs-convert:

## At this point you are done with the main steps and you can go on
## with the following optional steps:

## Optional but recommended steps taken from the man page of btrfs-convert:
## run defragmentation on the entire filesystem. 
## This will attempt to make file extents more contiguous.

sudo btrfs filesystem defrag -v -r -f -t 32M /
sudo btrfs filesystem defrag -v -r -f -t 32M /home

## this next steps is to  compact btrfs metadata. TAKES LONG!!
sudo btrfs balance start -m /
sudo btrfs balance start -m /home

## optionally delete the ext2 metadata
sudo btrfs subvolume delete /ext2_saved

If you installed on a SSD or NVME, enable fstrim.timer as both fstrim and discard=async mount option can peacefully co-exist:

sudo systemctl enable fstrim.timer

Again, for SSD trimming to work properly, it is important that you add discard to your crypttab (see above). Also check whether you find issue_discards=1 in /etc/lvm/lvm.conf (which should be correct by default):

sudo grep "issue_discards" /etc/lvm/lvm.conf 
#     # Configuration option devices/issue_discards.
#     issue_discards = 1

I recommend installing and the configuring to your needs btrfsmaintenance

sudo apt install -y btrfsmaintenance

you can then go and edit /etc/default/btrfmaintenance to suit your needs.

At this point you can install timeshift and timeshift-autosnap-apt and you are done (assuming you want to use timeshift for your snapshots, of course).

Linux Pop_OS