Setting up RAID1 with btrfs

Apr 10, 2021 Data Storage Script System

I do a lot of software development, and store all my code, media (video, audio, digital photos, etc) on a NFS server hosted in my network. Although I have a great backup strategy combining additional drives, rsync & restic, I was missing two main features: RAID and efficient snapshots. I used to have a basic form of snapshots using rsnapshot, however with literally over 1,000,000 website files (not to mention all the media), rsync is simply not a practical tool to use every hour of every day, even with a SSD.

In September 2020 I discovered btrfs, a “modern copy on write (CoW) filesystem for Linux aimed at implementing advanced features while also focusing on fault tolerance, repair and easy administration”. Its main features and benefits are snapshots, RAID & self-healing, exactly what I was looking for.

After a fair bit of testing, I switched several of my storage drives on my file server from ext4 to btrfs, and set up a separate RAID1 for two of the 2TB HDD drives I have.

Creating a RAID1 storage

There are a lot of tutorials online for setting up hard drives & various RAID systems with btrfs, including the official documentation. I’m just documenting the method I used to net up my RAID1 for future reference, and to help anyone else along the way.

For this example, I have two blank drive partitions (the same size) /dev/sde1 & /dev/sdf1 which I want to set up with RAID1, and mount to /mnt/raid.

# create the btrfs filesystems
mkfs.btrfs /dev/sde1
mkfs.btrfs /dev/sdf1

# mount the first drive
mount /dev/sde1 /mnt/raid

# add the second drive - resulting in combined storage capacity spanned across both drives
btrfs device add /dev/sdf1 /mnt/raid -f

# create RAID1 of data and metadata (important in case one drive fails)
# depending on your drive size, this can take several hours to complete
btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt/raid

Set the mount point in your /etc/fstab with the UUID of your RAID which you can see in blkid --match-token TYPE=btrfs

UUID=<UUID>    /mnt/raid btrfs    defaults,autodefrag    0 2

At this point /mnt/raid/ is a RAID1 using /dev/sde1 & /dev/sdb2. Any file you write or edit is automatically saved to both mirrored drives.

Rotated volume snapshots

The btrfs volume snapshot system is very efficient & incremental, meaning that only file changes are stored, everything else just points to existing files until they change. This means that you can have many snapshots of your filesystem with minimal additional storage. The amount of additional storage required for snapshots is completely dependent on the number of modified or deleted files.

Snapshots are described in detail on the btrfs website, and are, as you will see, very simplistic and do not deal with rotation. There are a number of apps & scripts listed on that page to help manage snapshot rotation. I couldn’t find one I needed, so I forked one I liked and made some adaptions which you can get here:

Comments