Installing ZFS on Root in a Running VPS

This guide explains how to install ZFS as the root filesystem of a VPS that contains an existing Linux installation without causing data loss. The VPS will have a small EXT4 boot partition for GRUB2 with the kernel and initrd, and a large ZFS pool mounted on root with compression enabled. Most commands will be issued from a live CD image and a workaround will be shown if the VPS does not support mounting ISO images.

Index:

1. Back up the data

2. Boot a live OS image

3. Install ZFS on the live OS environment

4. Partition the VPS disk

5. Install ZFS on the VPS disk

6. Restore the backup to the new ZFS root filesystem

7. Install GRUB2

8. Boot into the ZFS root pool

Step 1: Back up the data

If you have a second VPS or home computer with enough space to store the backup data, from within that computer run:

# rsync -aqrxz root@target:/ backup.d

where target is the hostname of the VPS being converted. This will copy the entirety of the root filesystem over SSH to the backup machine, and restoring this backup onto the target VPS later on will be straightforward. If you need to compress the backup for lack of space, you can alternatively create a compressed TGZ archive:

# ssh root@target "tar -cf - --acls --xattrs --one-file-system --absolute-names /" | gzip -1 >backup.tgz

However, when restoring a root filesystem from a TAR archive, it will be necessary to recreate all symlinks whose targets have absolute pathnames, because TAR converts their targets to relative pathnames, so using rsync is the simpler alternative. How to properly restore the TAR archive will be shown later.

Step 2: Boot a live OS image

The conversion to ZFS will be done from a live CD operating system. Depending on how capable your VPS control panel is, it may already have a live OS image ready to boot. Alternatively, you can download a Debian live ISO from the Debian mirror and upload it to your VPS control panel, then boot the ISO from there; this will work fine even if your VPS runs a different distribution. Be careful to use a live OS ISO image, not an installation ISO, because installation images lack apt and are unable to install the necessary software onto the live environment. If your VPS control panel supports booting from a user-supplied ISO image, skip to Step 3, otherwise you can still boot the downloaded ISO by converting a swap partition, if present, into EXT4 and storing the ISO in that reclaimed storage space, then GRUB2 may be configured to boot from that ISO. This is done as follows:

# fdisk --list /dev/vda
Disk /dev/vda: 35 GiB, 37580963840 bytes, 73400320 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x495b5ce4

Device Boot Start End Sectors Size Id Type
/dev/vda1 * 2048 69203583 69201536 33G 83 Linux
/dev/vda2 69203584 73397887 4194304 2G 82 Linux swap

This VPS has a 2GB swap partition in /dev/vda2 which is large enough to store the ISO. Create the EXT4 filesystem, mount it and download the ISO on it:

# swapoff /dev/vda2 # disable swap partition
# mkfs.ext4 /dev/vda2 # create EXT4 filesystem
# mount /dev/vda2 /mnt # mount the reclaimed space
# mkdir /mnt/images/
# wget -O /mnt/images/image_file.iso <image_URL>

then instruct GRUB2 to boot off this ISO; edit /etc/grub.d/40_custom as follows:

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.

menuentry "Live ISO" {
set ISOFile="/images/image_file.iso"
loopback loop (hd0,2)$ISOFile
linux (loop)/live/vmlinuz boot=live findiso=$ISOFile
initrd (loop)/live/initrd.img
}

and apply the configuration with

update-grub2

When you reboot the VPS, the GRUB2 menu will offer the option to boot off the live image. Proceed from there using the VNC rescue console offered by the provider's control panel.

Step 3: Install ZFS on the live OS environment

Boot the live OS image and access its terminal. Then install the ZFS packages on the live OS environment:

# echo deb http://deb.debian.org/debian stretch contrib >> /etc/apt/sources.list
# apt update
# apt install zfs-dkms
# modprobe zfs

Now the live OS has the ZFS kernel module and the ZFS utilities required to format your VPS root filesystem.

Step 4: Partition the VPS disk

There are two alternative scenarios for partitioning the VPS disk:

1. If you stored the live ISO in /dev/vda2 you may not alter that partition now because the live system depends on it, and no partitioning whatsoever will be done to the VPS disk. ZFS will be formatted on the existing /dev/vda2 partition, and /dev/vda1 will remain as is and later be repurposed as your permanent boot partition for storing the kernel and initrd. Skip to Step 5.
2. If your live environment does not depend on an ISO file stored on the VPS disk, repartition the disk by creating one small boot partition for GRUB2, and one large partition for the ZFS root that extends over all remaining disk space. Execute fdisk to create the two following partitions:

   # fdisk /dev/vda
   
   Command (m for help): p
   
   Disk /dev/vda: 35 GiB, 37580963840 bytes, 73400320 sectors
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   Disklabel type: dos
   Disk identifier: 0x495b5ce4
   
   Device Boot Start End Sectors Size Id Type
   /dev/vda1 2048 69203583 69201536 33G bf Solaris
   /dev/vda2 * 69203584 73397887 4194304 2G 83 Linux
   

   Create a primary boot partition 100MB wide and mark it as bootable with an asterisk (*), with a partition ID = 83 (Linux). It will be formatted as EXT4 for compatibility with GRUB2, and it will contain the boot data. Then create another primary partition for ZFS, with ID = bf (Solaris). Write the changes to disk and close fdisk. Now that the disk is properly partitioned, you may format the boot partition:

   # mkfs.ext4 /dev/vda2
   

Step 5: Install ZFS on the VPS disk

Format /dev/vda1 as a ZFS volume:

# zpool create -o ashift=12 \
-O acltype=posixacl -O canmount=on -O compression=zstd \
-O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
-O mountpoint=/ -R /mnt \
rpool /dev/vda1

This set of options is recommended by the ZFS documentation for the highest compatibility with the newest and future disks. The option compression=zstd enables the ZSTD compression algorithm which has a higher compression ratio than the default LZ4, although you may use compression=lz4 to reduce the CPU usage if your VPS has high disk I/O. The pool is named rpool which is the most commonly used name for root pools. When this command completes, rpool will be mounted on the /mnt directory of the live environment, as specified by the -R /mnt option above, but it will be automatically mounted on / at every subsequent boot. Mounting it on /mnt is necessary to restore your backup onto it from the live environment.

Step 6: Restore the backup to the new ZFS root filesystem

Install the OpenSSH server on the live environment to receive the backup data from the backup VPS:

# apt install openssh-server

For simplicity, enable PermitRootLogin yes in /etc/ssh/sshd_config of the live environment, then set a secure root password and restart OpenSSH:

# passwd
# service restart ssh

and initiate the restoration of the backup data from the backup VPS, which will populate the ZFS root pool mounted on /mnt:

## from the backup VPS
# rsync -arxz backup.d/* root@target:/mnt/

or if you used TAR to create a backup archive, restore it with:

## from the backup VPS
# cat backup.tgz | ssh root@target "tar -C /mnt -zaxf -"

however as I wrote in Step 1, upon extraction, TAR converts the target path of absolute symlinks into relative paths; these symlinks can all be properly restored as follows. Note that this restoration is not necessary if you didn't use TAR to back up the root file system. From the backup VPS, list all symlinks contained in the archive:

## from the backup VPS
# tar -tvf backup.tgz | egrep -- '->' >list_of_symlinks.txt

then copy the list to the target VPS into /mnt, which now contains the root filesystem repopulated from the backup:

## from the backup VPS
# scp list_of_symlinks.txt root@target:/mnt/root/

From the target VPS, chroot into /mnt and execute this Bash script to restore all absolute symlinks:

## from the chroot environment
while read -r row; do
src=$(echo "$row" | grep -Po '(?<=[0-9]{2}:[0-9]{2} )[^ ]+')
tgt=$(echo "$row" | grep -Po '(?<= -> ).+')

echo "linking: $src -> $tgt"
ln -fs "$tgt" "$src"
done </root/list_of_symlinks.txt

Your root filesystem is now fully restored from the backup and may be booted, but you need a functioning bootloader first.

Step 7: Install GRUB2

If you are inside the chroot, exit it and return to the live environment. Then bind the virtual filesystems to /mnt and mount the boot partition:

# mount --bind /dev /mnt/dev
# mount --bind /proc /mnt/proc
# mount --bind /sys /mnt/sys
# mount /dev/vda2 /mnt/boot

You may choose to copy any pre-existing content from /mnt/boot/ over to the new boot partition. Chroot into the root pool, and from within it, install the ZFS initramfs hook, the ZFS kernel module, and the ZFS userspace tools required to generate the initrd:

# chroot /mnt
# apt install pkg-dev linux-headers-amd64 linux-image-amd64
# apt install zfs-dkms zfs-initramfs

then install the GRUB2 package:

# apt install grub-pc
# update-initramfs -u -k all

edit /etc/default/grub to set the ZFS pool as the root device, by modifying the GRUB_CMDLINE_LINUX variable:

## this is /etc/default/grub
GRUB_CMDLINE_LINUX="root=ZFS=rpool"

then apply the configuration:

# update-grub

and finally install GRUB2 to the MBR:

# grub-install /dev/vda

Step 8: Boot into the ZFS root pool

The system is ready, fully restored from the backup, and may be booted. Exit the chroot and umount all virtual filesystems, then export the ZFS root pool:

# exit # exit the rpool chroot
# umount /mnt/dev
# umount /mnt/proc
# umount /mnt/sys
# umount /mnt/boot
# umount /mnt
# zpool export -a

You can now remove the ISO image from the VPS control panel, if you used this option, or delete the ISO from the hard drive, and reboot the system into the ZFS root pool:

# reboot

The conversion of the VPS root filesystem from EXT4 to ZFS is complete, and the VPS will boot into your old installed system as usual.

Hosting a Minecraft Server: VDS or Laptop?

Hosting on a Laptop

Hosting a Minecraft server on your laptop seems like a convenient and cost-effective solution, especially if you already own a capable machine. This approach allows you to have immediate control over the server environment, making it easy to manage and configure settings directly. However, this convenience comes with some trade-offs.

One of the main drawbacks is performance. Running a Minecraft server can be resource-intensive, consuming a substantial amount of CPU, RAM, and bandwidth. This can significantly slow down your laptop, especially if it is not a high-end model. You might experience lag and potential crashes during intensive gameplay if your laptop's RAM is insufficient, which can be frustrating for both you and your players.

Reliability is another concern. Your server's uptime is directly tied to your laptop's stability and internet connection. Any power outages, hardware issues, or network disruptions can lead to unexpected downtime, disrupting the gaming experience. For example, imagine hosting a server for a group of friends. During an important in-game event, your laptop overheats and shuts down, causing frustration and disruption for all players involved.

Additionally, hosting a Minecraft server with network ports open to the world exposes it to security risks. If you plan to run a public host and let anyone into your server, you will need to implement robust security measures, such as firewalls and antivirus software, and protect against DDoS. These risks can be mitigated by configuring the server as private and restrict access to anyone but the friends you trust.

Hosting on a VDS

In contrast, hosting a Minecraft server on a VDS offers several compelling benefits that can enhance the overall gaming experience. VDSs are designed to handle server loads, providing better performance and stability. This means you can expect smoother gameplay with minimal lag, even during peak times.

One of the key advantages of using a VDS is reliability. VDS providers often have redundant power and network connections, ensuring higher uptime and reliability. This means your server will be available to players more consistently, without the risk of unexpected downtime due to hardware or network issues.

Scalability is another significant benefit. With a VDS, you can easily upgrade resources such as CPU, RAM, and storage as your server grows. This flexibility allows you to accommodate more players and larger worlds without compromising performance. Additionally, VDS providers typically offer robust security measures and regular backups, protecting your server from potential threats.

Network latency is another important factor to consider. VDS hosting often provides better connectivity and lower latency compared to a personal laptop. This is because VDS providers usually have high-speed internet connections and are located in data centers with optimized network infrastructure. Lower latency means faster response times and a smoother gaming experience for players, especially those connecting from different geographical locations.

For instance, hosting a public server on a VDS allows you to support a larger player base with consistent performance. Even during peak times, the server remains stable, providing a seamless experience for all players. This level of reliability and performance is difficult to achieve with a personal laptop.

While VDS hosting comes with a monthly fee, the investment is often justified by the enhanced performance, reliability, and security it provides. Moreover, setting up and managing a VDS requires some technical knowledge, but many providers offer managed services to help with the initial setup and ongoing maintenance.

Final Thoughts

In summary, while hosting a Minecraft server on a personal laptop can be convenient and cost-effective, it comes with limitations in terms of performance, reliability, and security. On the other hand, using a VDS offers superior performance, higher reliability, and robust security measures, making it a compelling choice for Minecraft enthusiasts. By carefully weighing the pros and cons, you can make an informed decision that best suits your needs and ensures an optimal gaming experience for you and your players.

RackNerd Black Friday 2024 Offers

RackNerd has released four new VPS offers in anticipation of Black Friday 2024 named the 11.11 Specials. Let's have a look and compare prices and specifications and see who their direct competitors are.

RackNerd VPS List:

These are some of the 11.11 Specials just released by RackNerd, you can see the full list by executing ❯ this search query.

RackNerd's Competitors:

Now let's take a look at what RackNerd's direct competitors have to offer to match the same hardware specifications, and at what price point. Here ❯ you can see the full list. At the time of writing, ColoCrossing matches all of RackNerd's specifications and manages to strike a better price for their VPSs. Remember however that all monthly prices indexed in the database are obtained by dividing the annual prices by 12 for the purpose of data normalization.

The Ultimate VPS Directory and VPS List Comparison Tool

VPS Price Tracker is among the most comprehensive VPS Directories available online. With over 60,000 VPSs from nearly a hundred providers, this search engine aggregates and indexes a vast array of offers, making it easy to compare and choose the best VPS for your project. My mission is to help you find the perfect Virtual Private Server (VPS) that meets your specific needs and budget.

The user-friendly interface allows you to search for VPSs based on a multitude of parameters, including CPU class, memory size, disk type, port speed and bandwidth, and a customized VPS list is compiled just for you with the results of your search. This level of customization ensures that you can find the ideal VPS that aligns with your requirements.

Each VPS listing in the directory is presented in a compact product card, showcasing the most relevant information at a glance. This includes details such as monthly price, disk type and size, memory, CPU, and more. Additionally, you can view the complete data for each VPS, including whether it supports IPv4, IPv6, and various virtualization types from the Info page.

VPS Price Tracker is designed to be the reference point for all your VPS needs. Whether you're a seasoned developer or just starting out, this comprehensive VPS directory will help you make an informed decision and find the perfect VPS for your project.

Familiarize yourself with the search tool on the sidebar and discover the ideal VPS for your needs. With this extensive database and powerful search engine, you're sure to find the perfect VPS that meets your requirements and budget.