[Tip] Root filesystem nearly full but got a spare partition sitting around? Split off some stuff!

Difficulty: ★★★☆☆


Introduction

As you should all already know by now, GNU/Linux is a FLOSS variant of the UNIX operating system design, and UNIX does not know the concept of drive letters, nor does it approach storage from the vantage of having different volumes. Instead, everything is mounted into a uniform directory hierarchy, so that regardless of what physical medium any particular group of files resides on ─ even if this physical medium is actually located in another computer across the network ─ you will always be able to navigate to said files by way of the directory structure, because they’ll always be in the same location of the hierarchy.

Now, in Microsoft Windows and even in Apple macOS ─ which, like GNU/Linux, is a UNIX system underneath its graphical user interface ─ it is custom to install the entire operating system into a single physical volume, or otherwise put, a single partition on an HDD or SSD. However, UNIX does not require this; it only requires certain things to reside on the root filesystem if they are needed at boot time. Everything else can in essence be split off and put on a separate filesystem.

Now, up until a number of years ago, all the stuff that was needed for booting ─ i.e. the contents of /bin, /sbin and /lib ─ needed to reside on the root filesystem. But with the advent of systemd, and with support for more and more hardware being added to the system libraries, this made the root filesystem keep growing bigger and bigger.

As such, the decision was made by certain distributions to move all of that stuff out to the equivalently named directories under the /usr hierarchy. But this in turn meant that the contents of /usr would still need to reside on the root filesystem, because all of the stuff needed at boot time or in single-user maintenance mode was now all parked under /usr.

Thankfully, ever since the beginning the Linux kernel has allowed booting with the help of an initial RAM filesystem, the initramfs, which can contain all of the driver modules and tools needed for booting. And so now the contents of /usr can be put on a separate filesystem again, and if /usr is being shared across the network, then this filesystem can be mounted read-only without that the whole of the root filesystem would need to be mounted read-only as well.

The bottom line is that, unlike in Microsoft Windows, a UNIX operating system can actually be installed in a distributed manner across multiple filesystems, each of which can have its own dedicated mount options for added security, performance and stability.

Now, with the above all said, not everything can be split off from the root filesystem ─ there are still some things that need to reside on the root filesystem itself ─ but a whole lot more can be split off than just /home.

The reason why I’m going to be addressing that topic in this thread is that we often come across people here on the forum who are running out of space on their root filesystem. Commonly, these are people who until recently were dual-booting with Microsoft Windows and who are looking to either delete their Windows partition and extend their GNU/Linux root partition, or who have already reformatted their former Windows partition with ext4 or a similar Linux-native filesystem, but who don’t know how to put this recovered storage to good use.


What can be split off from the root filesystem?

The contents of all of the following directories can all be physically located on separate filesystems… :arrow_down:

/boot
/home
/opt
/srv
/tmp
/usr    (requires some extra work in Manjaro ─ see farther down)
/var

Now, a number of directories is already split off from the root filesystem by default in Arch and Manjaro, namely… :arrow_down:

/dev          : resides on devtmpfs, a special virtual-memory-based filesystem
/tmp          : resides on tmpfs, a virtual-memory-based filesystem
/proc         : is a virtual filesystem ─ it is an interface to the kernel
/sys          : is a virtual filesystem, through which the kernel exports information about the hardware
/run          : resides on tmpfs, a virtual-memory-based filesystem

In other words, the above directories are not consuming any disk space.


Okay, my root filesystem is almost full, and I do have a spare partition. Now how do I go about it?

Well, the first thing you need to do is check how big the spare partition is, so that you’d know what you can split off. And then next, you can use the du command to find out which directories are using up the most space on your root filesystem.

Without a doubt, /usr will normally be the biggest one, but splitting off the contents of /usr is a special case ─ which I will be getting into farther down ─ due to the fact that some of the contents of /usr are needed at boot time.

Therefore, by way of an example, we’re going to split off /home, and I will be explaining the extra steps needed for splitting off /usr farther down. This way, you’ll be able to use the methodology for splitting off /home with any other directory ─ e.g /var, /opt, et al.

Getting your hands dirty

The first thing you need to do is ascertain the UUID of the spare partition. I am also going to assume for the remainder of this post that you’ve already formatted the spare partition with ext4, because that is something you can do from within a GUI partition manager.

Open up a terminal window and enter the following command… :arrow_down:

lsblk --tree -o NAME,UUID

Take a piece of paper and write down the UUID of the spare partition. Remember that UNIX is case-sensitive, so an uppercase “A” is not the same thing as a lowercase “a”. Make sure you get that UUID correctly!

Next, you have to completely log out of your GUI environment. This will close a larger number of open files and system libraries, and that’s what we need.

When the login screen reappears, press Ctrl+Alt+F3 to switch to a character-mode virtual console, also known as a tty. Log in at the tty and enter the following command… :arrow_down:

sudo systemctl isolate rescue.target

The use of sudo will require you to type your regular password. However, as soon as you’ve entered that command, the system will present you with a message that prompts you to either enter the root password or press Ctrl+D to continue. This is where you must enter the root password, which may be different from the password you normally use with sudo.

If you’ve entered the root password correctly, then the system will now present you with a root command prompt, and you will now be in single-user maintenance mode. This means that all multiuser activity has been terminated, and that the network is down. This is the safest way for doing what you’re about to do.

Now we are going to temporarily mount the spare partition into the tree. For the sake of ease, I’ll assume that the spare partition is /dev/sda5, but substitute this by whatever the device special file is that corresponds to your spare partition, e.g. /dev/nvme0n1p5. :arrow_down:

mount -t ext4 /dev/sda5 /mnt

Verify that you’ve got the right partition ─ the following command should come up with zero results. :arrow_down:

ls -lA /mnt/

Now we are going to copy over the contents of the directory that you wish to split off from the root filesystem, in this case, /home. :arrow_down:

cp -RPpv /home/* /mnt/

This is going to take quite some time, but it’s verbose, so you should see all of the files and directories being copied over. Let the command finish. Maybe get yourself a cup of coffee. :wink:

When the command has finished and the prompt returns, check whether everything is there… :arrow_down:

ls -l /mnt/

Maybe, just to be sure, check whether the used space is the same as in the source directory. :arrow_down:

du -sh /mnt/
du -sh /home/

The output of both commands should be the same.

Now, grab the piece of paper upon which you wrote the UUID of the spare partition, because now we are going to add a record for the new filesystem to /etc/fstab. :arrow_down:

nano /etc/fstab

Add a new blank line to the file, and then fill in that line as follows… :arrow_down:

UUID=the-UUID-you-wrote-down   /home   ext4   auto,defaults     0   0

If the new partition resides on an SSD, then you will also want to add ssd,noatime to the mount options ─ remember: no spaces behind the commas! :wink:

Save the file with Ctrl+O followed by Enter/Return, and press Ctrl+X to exit the editor.

Now you can clean up and reboot. :arrow_down:

sync
rm -rf /home/*
systemctl reboot

If the directory you wish to split off is /usr

As I said higher up already, this one requires a few special measures, because /usr is needed at boot time, and it is the directory that (via its subdirectories) holds the executables you’re using, even right now in single-user maintenance mode.

Without going through the whole process again, you can apply all of the steps explained above up until editing /etc/fstab. Of course, the /etc/fstab line for /usr will look a bit different. :arrow_down:

UUID=the-UUID-you-wrote-down   /usr   ext4   auto,defaults,sync     0   0

Same comment as for /home if the new /usr partition resides on an SSD, i.e. add the necessary mount options.

Save the file with Ctrl+O followed by Enter/Return, and press Ctrl+X to exit the editor.

Now here we have to introduce a few extra steps. Given that /usr is needed at boot time, you’re going to have to add a couple of HOOKs to /etc/mkinitcpio.conf and rebuild the initramfs. :arrow_down:

nano /etc/mkinitcpio.conf

Look for the line that starts with… :arrow_down:

HOOKS=(base udev ....

Insert the word usr behind the word udev and before the word autodetect, with proper spacing. Then, at the end of the line, insert the two words fsck and shutdown right before the closing ellipsis.

Alternatively, I myself am using systemd instead of udev, so my HOOKS line is a bit different. The advantage of this is that the systemd hook is actually more efficient and will yield a slightly faster boot ─ on my system here, it shaved off one second from the already fast boot time. It also foregoes the need to add the usr hook, because systemd will detect that automatically when the initramfs is rebuilt.

This below is my HOOKS line. Note that it also doesn’t use the keymap hook ─ it is replaced by sd-vconsole, which works with systemd ─ and that I don’t have the fsck added, which is because I’m using btrfs instead of ext4. :arrow_down:

HOOKS=(base systemd autodetect modconf block keyboard sd-vconsole filesystems shutdown)

Whichever solution you choose, save the file with Ctrl+O followed by Enter/Return, and press Ctrl+X to exit the editor.

Now that we’ve modified /etc/mkinitcpio.conf, we must rebuild the initramfs. :arrow_down:

mkinitcpio -P

This will rebuild the initramfs for your current kernel, as well as for the “Fallback” option in the GRUB submenu.

And now, all we still have to do is clean up, albeit in a slightly different way compared to the example with /home higher up. However, only do this if everything you’ve done so far was successful! :arrow_down:

PATH=${PATH}:/mnt/bin
update-grub
sync
rm -rf /usr/*
systemctl reboot

Epilogue

If everything went as planned, then your system will now come up on reboot just as it has before, but your root filesystem will now have a lot more free space on it, and whatever directory you split off, it’ll still be in the same place in the directory hierarchy. So, /home is still /home, /usr is still /usr, and so on. :wink:


Disclaimer

The warranty expired at the moment you started reading this. If it breaks your system, then you get to keep all the pieces.

:smiling_imp:


And always remember… :arrow_down:

Gollum_Backupses

12 Likes

Nice guide but why to suggest cp (and latyer on a very dangerous rm -rf) instead of mv?
(I think is better, and if anything maybe check disk space or permission in advance)

mv, when used between different filesystems, is the same thing as cp followed by rm -f, but then you don’t have any way of verifying that the transfer was complete. The way I have shown it in the post above at least gives you that option.

2 Likes

What about rsync ?

  • moves or copys, and detects if it is on the same drive
  • is very fast, and shows progress (if you want)
  • checks with checksums if it succeeded
  • can be interrupted and resumed
  • can handle permissions as you want

rsync is an option, but is a more complex tool to use for newbies than cp. One could also move the files over with tar, but again, that’s too complex for newbies. :wink:

2 Likes