In our previous article we described an idea setup for a modern server with btrfs for flexibility and redundancy. In this article we describe another kind of setup that is ideal only for a backup server. For a backup server redundancy and high availability are not important, but instead maximal disk space capacity and the amount of backups it can hold. Additionally it is good if the disks are encrypted to protect agains physical theft.
In this article we explain how to encrypt the disks with LUKS/dm-crypt so that all files are encrypted, and how all disks can be automatically opened using one single password entered by the administrator at boot time, as having multiple passwords would just add complexity without any gain, since all passwords are equally strongly protected by the same encryption technology.
Use Ubuntu installer to get the basic LUKS/dm-crypt in place
The easiest way to get started and get all pieces of software, Grub modules and other in place is to run the Ubuntu installer and install on a single disk a standard system using the option Encrypted disk. This will deploy an unencrypted /boot partition, and an encrypted LUKS/dm-crypt, LVM and a ext4 / (root) partition and an encrypted swap partition on the first disk.
Once booted into this freshly installed system you can start tweaking it.
Create one big encrypted btrfs filesystem as /data
Next we will explain how to encrypt additional disks and how to use btrfs to glue multiple differently sized physical disks together for maximal disk space, and how to use btrfs snapshots to enable as much historical backups as needed with minimal additional disk usage.
The following commands dmsetup ls --tree and dmsetup status will list what devices Device Mapper knows about. To view all physical devices and the block devices they map to, the tool lsblk is also excellent. The following command will show information about the encryption:
$ cryptsetup -v luksDump /dev/sda5
LUKS header information for /dev/sda5
Version: 1
Cipher name: aes
Cipher mode: xts-plain64
Hash spec: sha256
Payload offset: 4096
MK bits: 512
MK digest: aa bb cc ddaa bb cc dd ee ff aa bb cc dd ee ff aa bb cc dd
MK salt: aa bb cc dd ee ff aa bb cc dd ee ff aa bb cc dd
aa bb cc dd ee ff aa bb cc dd ee ff aa bb cc dd
MK iterations: 37500
UUID: 6507313f-d061-4150-9ca2-7966b17cdce2
Key Slot 0: ENABLED
Iterations: 149532
Salt: aa bb cc dd ee ff aa bb cc dd ee ff aa bb cc dd
aa bb cc dd ee ff aa bb cc dd ee ff aa bb cc dd
Key material offset: 8
AF stripes: 4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: ENABLED
Iterations: 153845
Salt: aa bb cc dd ee ff aa bb cc dd ee ff aa bb cc dd
aa bb cc dd ee ff aa bb cc dd ee ff aa bb cc dd
Key material offset: 3536
AF stripes: 4000
Command successful.
This is the encrypted device created by the Ubuntu installer. There is one single device, which Ubuntu assigns as a LVM device, and then splits into a root and swap partition.
sda 8:16 0 232,9G 0 disk ├─sda1 8:17 0 487M 0 part /boot └─sda5 8:21 0 232,4G 0 part └─sda5_crypt 252:0 0 232,4G 0 crypt ├─ubuntu--gnome--vg-root 252:1 0 230,5G 0 lvm / └─ubuntu--gnome--vg-swap_1 252:2 0 1,9G 0 lvm [SWAP]
This can be left as is, and we will create a new /data partition for our btrfs filesystem using multiple encrypted hard disks.
Encrypted disks
After adding more physical disks to the server, boot it and run the following commands to get the software part done.
First format all disks with fdisk so you have something like this:
$ fdisk -l /dev/sdb Disk /dev/sdb: 931,5 GiB, 1000204886016 bytes, 1953525168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x8a63d658 Device Boot Start End Sectors Size Id Type /dev/sdb1 * 2048 1953525167 997376 999G 83 Linux
Then create a keyfile containing some random data. The keyfile will be stored on the encrypted system disk and used to open the encrypted drives.
dd bs=512 count=4 if=/dev/urandom of=/etc/cryptkey iflag=fullblock
This makes the boot process smooth: you only need to enter a password to unlock the system disk, and after that all other disks will open and mount automatically.
Then format the new partitions as encrypted LUKS devices that can be opened using the one and same keyfile:
cryptsetup -s 512 -v luksFormat /dev/sdb1 /etc/cryptkey cryptsetup -s 512 -v luksFormat /dev/sdc1 /etc/cryptkey
Then test that you can open the devices with the command:
cryptsetup open --key-file=/etc/cryptkey --type luks /dev/sdb1 wdc1tb
The last parameter defines which name the device should have when presented by Device Mapper. In my naming scheme I use the disk brand (Western Digital) and size (1TB) so that it is easier later to somehow map which physical disk is which partition.
To have the devices opened automatically at boot, add to /etc/crypttab
wdc1tb /dev/sdb1 /etc/cryptkey luks,noearly wdc1tc /dev/sdc1 /etc/cryptkey luks,noearly
A LUKS device can have multiple keys. There are up to 8 key slots available.
A new key for slot 7 on device sdb1 can be added like this:
sudo cryptsetup luksAddKey --key-slot 7 /dev/sdb1 -d /etc/cryptkey
Once dm-crypt/LUKS devices are set up and opened, the system will see a new block device, which you can format to btrfs with:
Create a btrfs filesystem
mkfs.btrfs -m raid1 -d raid0 /dev/mapper/wdc1tb1 /dev/mapper/wdc1tc1
Add as many devices to this command as you like. You can also later add more devices with ‘btrfs device add’. With these parameters the disks will be joined to form one big logical paritition with the size of all disks combined (RAID0). Only metadata is duplicated (RAID1).
Finally mount the new btrfs filesystem with:
mount -t btrfs -o defaults,noatime,compress=lzo /dev/mapper/wdc1tb1 /data
After this you’ll have at /data a big btrfs filesystem. Now you can start making your backups to /data using your preferred backup software. For most cases, simple rsync is enough.
Using snapshots to get some backup history
To make a read-only snapshot of all data in /data, run:
btrfs subvolume snapshot -r /data /data/snapshot.$(date +%Y-%m-%d)
Delete a snapshot with: btrfs subvolume delete snapshot.2017-08-12
To show how much disk space is used, run: btrfs qgroup show. Unfortunately on certain systems using quota groups makes the btrfs-cleaner run amok, so only enable quotas if your system is not affected by btrfs bug #116801.
Using external USB disks?
The default settings in Ubuntu are set to turn off external USB disks after a certain time of inactivity. This can be harmful is the disks are used as a part of an btrfs array. Disable autosuspend by running:
for i in /sys/bus/usb/devices/*/power/autosuspend; do echo 2 > $i; done
for i in /sys/bus/usb/devices/*/power/level; do echo on > $i; done
How to add more disks
If you ever need to extend the setup, you need to remember to:
- Partition the new disk with fstab.
- Encrypt the new disk with cryptsetup.
- Mount the new encrypted device with
cryptsetup open .... - Add it to the the btrfs pool with
btrfs device add /dev/mapper/... /data - And finally update /etc/crypttab so that at boot the encrypted device is automatically mounted. Btrfs does not need any fstab changes as the pool will detect its members automatically.
One thought on “Secure and flexible backup server with dm-crypt and btrfs”