Difficulty: ★★☆☆☆
Content table
I am experimenting with including this for longer articles. Provide feedback if it was useful!
Preface
Preparation
Assumptions
Dual-boot
File systems
Media configuration
Execution
Enabling the common space
Using the common space
Why even?
Preface
Beyond the simple act of making another partition for /home during installation, there may be instances where a user may wish to have files accessible elsewhere, on another operating system with no direct support of the ext4 format as a common space used by multiple machines and systems. Whether you rock a legacy board with BIOS or a modern board with UEFI, these tips can help to save your precious work from being destroyed, and be a bit more organized.
How partitions mounted at startup are handled may differ between desktop environements / shells. As I use MATE for my desktop environment, I generally have no issues or extra provisions to worry about for the tricks below to be functional.
Preparation
Ideally, you should consider this before installing the system as retrofitting it into an existing deployment will take much, much longer depending on length of use and files involved.
Encryption won’t be mentioned here, but if you want to give it a try, good luck! Most of these configurations may not be so encryption-friendly.
Assumptions
For brevity, I am going to assume this: You were already going to create a dual-partition setup, or you have an existing Windows system you want to access the same files from as on your Linux system, and you want to limit operations to /home as much as possible for productivity.
Linux system configuration
Additionally, I am going to assume the following partition sizes if they do not exist already, as these haven’t failed me yet with strict organization discipline:
- 32GB for
/(root, system) - 16GB for
/home(your stuff)
Learn why these figures were used
Why so small? Because the whole point of this is to not keep your files in
/home. Rather, directories will be referenced from$HOMEinto another mounted partition. Multiple users? I would allocate another 4GB per extra user in a household for insurance. Config files aren’t that big and so long everybody agrees on disciplined file organization there shouldn’t be any problems with this!
Some people expressed concerns about the small system size. In a worst-case scenario there may be less than 100GB to spare for someone to trial and use long-term an open-source system, and personally I found so long conventional packages and shared libraries are used, 32GB is good enough, but if it is not then users are free to make their root partition larger to suit their needs.
Another major advantage to using smaller system partitions, so long they are not encrypted is the idea of making an installation portable. Open-source systems can operate without fatal issues via USB 2.0 (but faster is always better) and with the above figures, a duplicate installation can be stored on a 64GB USB-attached media, with plenty of space for important effects as needed!
Dual-boot
If you are a current Windows user, expanding this may be of interest to you.
Moving effects
If using Microsoft Windows, moving your personal effects off of the disk is preferable for shrinking Windows as small as possible (with space for updates), but considering the difficulty of separating effects from the system compared to most open-source solutions, the best you can do is copy the “Big five” library directories from %userprofile% to somewhere else.
To see how this is possible, see Execution / Using the common space.
If it isn’t tenable to resize the partition Microsoft Windows is on for inclusion of parts to install an open-source system, then provision an external media to store both the open-source system and personal effects. This way, you can still access your stuff and use the open-source system on any machine you have use rights for, so long library files for hardware compatibility are installed in advance.
File systems
Whatever your configuration is, it should be suited to the systems you intend to access with. ext4 is a given, and supports everything you need but doesn’t mount on Microsoft Windows, Apple or Android. Apple’s apfs is proprietary, and hfs+ has a reputation to incur bit rot over time (of which, the validity of such claims won’t be examined here). There are two safe options (courtesy of Microsoft) you can use as a common space for your personal effects:
-
ntfs : Microsoft’s standard partition format for Windows which most mainstream Linux systems can read. Uses file ID system rather than a hard limit of file index nodes. While typically only the domain of Windows, this format supports symbolic links with ext4and in both directions. -
exfat
: Microsoft’s extended file allocation table format which allows for more than 4GB per file, versus fat32which requires splitting files past this hard limit. Practically usable across modern systems, but does not support symbolic links coming from it.
If you need to access the same data across Android, OS X, Windows and open-source systems, then exfat becomes your only option. If sharing data in a common space between your open-source OS and Microsoft Windows alone, then you can use ntfs without any issues.
ntfs: I forgot to mention this, but if power down was disgraceful, once in awhile the partition will be in a read-only state. If this happens, perform the following:
- Reboot into Windows
- Run
chkdskon affected partition- Reboot into Windows once more
- Power down completely
- Boot into open-source instance
- Run
sudo ntfsfix -d /dev/sdXN
Replace
/dev/sdXNwith device and partition as shown withlsblk -f.Eventually you’ll clear the read-only state and new content can be saved onto the
ntfspartition.
exfat: This partition format is not nearly as safe compared tontfsand it is also much harder to resize this partition format without the use of third-party tools on Windows. To prevent excessive data corruption, perform a scan withchkdskin a Windows session occasionally.
Media configuration
About limited partitions
Most recent storage media devices are able to support more than four partitions, however if your options for partitions are limited, open-source systems can operate comfortably inside of an extended partition space, where an additional one megabyte is used per partition.If there are recovery or diagnostic partitions for your particular system, don’t delete those. Try to work around them, and if it isn’t possible to create new partitions due to media limitations then duplicating the EFI partition to another device and making said duplicate bootable can allow for deleting the original EFI partition to create an extended partition. This is an advanced procedure which will not be covered in this topic with any depth beyond this text.
Regardless whether or not other partitions existed prior to installation, the whole point of this is to not use an ext4 partition for your stuff. Without going into any degree of depth about the installation process, it is assumed from hereon all partitions you need for system and storage of personal effects had already been configured in gparted / kparted or gnome-disks.
Needless to say, none of the automated options in Calamares supports any of what this topic covers, so you should be comfortable as an end-user to exercise more agency over how your media is arranged and manually configure when prompted for which installation method to use.
Execution
After installation of the system is finished, or if retrofitting this advice by shoehorning other partitions into an existing setup, there are a few operations which need to be performed for use of this common space established to store your personal effects.
I have no experience with OS X, so I have nothing for those users. If using an OS X system in tandem as a hackintosh or compatible PowerPC system, how to resolve this should come naturally using the other advice below.
Enabling the common space
Windows
Microsoft Windows will, by default mount all connected devices and enable all partitions on attached media by default. There is no further action necessary for this to work.
Linux
This assumes
gnome-disk-utilitypackage is installed.
For ease of use, there is no better software I can think of than GNOME Disks. Upon execution, you’ll notice the left pane with all attached media, and the right pane shows details about the selected attached device.
Select the device your intended common space (and probably also, system) is on and in the graph bar, the partition where your effects will be / are at. Click on the gears symbol beneath the bar graph and select Edit Mount Options…
From this dialogue, disable User Session Defaults and toggle on Mount at system startup. Optionally, change via the Identity dropdown how the partition is identified according to /etc/fstab. That is all which is necessary. If for some reason you don’t want to use gnome-disks, here is a generic /etc/fstab entry you modify to suit:
insert with correct device and partition in
/etc/fstab:<IDENTITY> <MOUNTPOINT> auto nosuid,nodev,nofail 0 0
There are a variety of ways to handle this based upon available information, so it is best to read the manual for
fstabin a terminal emulator before using the generic example above.
When finished, click on the play button below the bar graph and the partition will be mounted. WIth subsequent reboots, this partition will be mounted automatically.
Using the common space
Method of access and partition used are mutually exclusive. While method of access may be favourable depending on partition format, both can be used for each, which is why using one method for a particular format will only be advised rather than strictly for that format.
For brevity, everything past this will assume use of the the big five directories:
DocumentsDownloadsMusicPicturesVideosAlso, we’ll assume the partition (when mentioned) is labelled
fooand this is already in/etc/fstab:In a hypothetical, example
/etc/fstab:LABEL=foo /mnt/foo auto nosuid,nodev,nofail 0 0For anything involving Windows, we’ll assume what is known as
/mnt/foo/isF:and the directory for two-level links is known asfoostuffsince dot-hidden directories are not a thing which is possible.
Expand the sections below depending on what you wish to use:
Windows
Registry
To do this in more modern builds of Windows (Vista+), simply right-click on a library directory in Explorer and see its Properties. There should be a tab labelled Target which allows you to define where the files are. If you have another USB-attached media which can accommodate space for all directories, then Explorer will put forth the effort on your behalf to move the files to the new location.
Once finished, resize the ntfs partition Microsoft Windows is on using diskmgmt.msc within the current session (as Windows allows this) and create the new common space which will become the target location for these library directories.
The benefit in doing it like this is that these directories will already be defined in the Windows registry so continued operations in an open-source system can take place, with less rebooting necessary.
Limited disk space
If resizing the Windows partition substantially is impossible — even after use of chkdsk and defragmenting the media
— then you may just want to keep your effects on whatever separate device you have. So long there is enough space for the open-source system partitions, the open-source system can still be installed.
If using
mklinkrather than defining library targets, don’t forget that utility does things backward compared to GNU coreutils’lnprovided by most open-source systems.
Symbolic links
If you would rather not muck with registry settings or use Explorer to define directory paths for library locations, then this can be done instead to make use of common space in Windows possible.
Use of two-level links
If for some reason the partition location or drive letter changes, the operations shown after this will have to be performed again. To prevent the need of recreating every link, the mountpoint could be linked instead as shown:The first of two levels
# Define links mklink /S %userprofile%\foostuff F:If incorporating this, all future links should begin with
%userprofile%\foostuffas this link directs to the same path asF:. If this changes, re-creation with the new path will re-establish all links:This time with
G:unlink %userprofile%\foostuff mklink /S %userprofile%\foostuff G:
To utilize the directory paths from /mnt/foo/ as symbolic links, the procedure below will enable access to the directories from there in $HOME.
# Copy because move could cause file corruption if interrupted. xcopy /E %userprofile%\Documents\ F: xcopy /E %userprofile%\Downloads\ F: xcopy /E %userprofile%\Music\ F: xcopy /E %userprofile%\Pictures\ F: xcopy /E %userprofile%\Videos\ F: # Delete originals pre-link rmdir /Q /S %userprofile%\Documents rmdir /Q /S %userprofile%\Downloads rmdir /Q /S %userprofile%\Music rmdir /Q /S %userprofile%\Pictures rmdir /Q /S %userprofile%\Videos # Define links mklink /S %userprofile%\Documents F:\Documents mklink /S %userprofile%\Downloads F:\Downloads mklink /S %userprofile%\Music F:\Music mklink /S %userprofile%\Pictures F:\Pictures mklink /S %userprofile%\Videos F:\Videos
Linux
XDG directories
While exfat doesn’t allow symbolic links from it, ext4 (and presumably, ntfs) allows symbolic links to directories in an exfat partition. Regardless, if using exfat it may be advisable to redefine XDG directories instead for avoiding trouble with links.
The following will do just that, changing the paths XDG uses for the big five directly:
Sample script for populating an
exfatpart and using XDG for home directories:# Copy because move could cause file corruption if interrupted. cp -R $HOME/Documents/ /mnt/foo/ cp -R $HOME/Downloads/ /mnt/foo/ cp -R $HOME/Music/ /mnt/foo/ cp -R $HOME/Pictures/ /mnt/foo/ cp -R $HOME/Videos/ /mnt/foo/ # Define paths xdg-user-dirs-update --set DOCUMENTS /mnt/foo/Documents/ xdg-user-dirs-update --set DOWNLOAD /mnt/foo/Downloads/ xdg-user-dirs-update --set MUSIC /mnt/foo/Music/ xdg-user-dirs-update --set PICTURES /mnt/foo/Pictures/ xdg-user-dirs-update --set VIDEOS /mnt/foo/Videos/ # Delete originals post-copy rm -rf $HOME/Documents/ $HOME/Downloads/ $HOME/Music/ $HOME/Pictures/ $HOME/Videos/
Yes, you read that right,
DOWNLOAD. Not plural.
Symbolic links
If you would rather not configure using XDG and find it easier to perform symbolic links regardless whatever partition format is in use, then below is the only other way to make this work.
Use of two-level links
If for some reason the partition location or identity changes, the operations shown after this will have to be performed again. To prevent the need of recreating every link, the mountpoint could be linked instead as shown:The first of two levels
# Define links ln -s /mnt/foo/ $HOME/.fooIf incorporating this, all future links should begin with
$HOME/.foo/as this link directs to the same path as/mnt/foo/. If this changes, re-creation with the new path will re-establish all links:This time with
/mnt/bar/unlink $HOME/.foo ln -s /mnt/bar/ $HOME/.foo
To utilize the directory paths from /mnt/foo/ as symbolic links, the procedure below will enable access to the directories from there in $HOME.
Sample script for populating an
ntfspart and using symlinks for home directories:# Copy because move could cause file corruption if interrupted. cp -R $HOME/Documents/ /mnt/foo/ cp -R $HOME/Downloads/ /mnt/foo/ cp -R $HOME/Music/ /mnt/foo/ cp -R $HOME/Pictures/ /mnt/foo/ cp -R $HOME/Videos/ /mnt/foo/ # Delete originals pre-link rm -rf $HOME/Documents/ $HOME/Downloads/ $HOME/Music/ $HOME/Pictures/ $HOME/Videos/ # Define links ln -s /mnt/foo/Documents/ $HOME/Documents ln -s /mnt/foo/Downloads/ $HOME/Downloads ln -s /mnt/foo/Music/ $HOME/Music ln -s /mnt/foo/Pictures/ $HOME/Pictures ln -s /mnt/foo/Videos/ $HOME/Videos
Links to common space
The use of symbolic links to common space is discouraged, though if for some reason you would like to perform this, understand that any links coming from the ext4 partition will be invalid if using Microsoft Windows as the physical system in session. Strict discipline to minimize use of links coming from ext4 will ensure a happier time if needing to use Microsoft Windows.
Why even?
If you scrolled to the bottom of this post for a tl;dr to the entire thread, put simply it’s this: Placing your stuff elsewhere is nice, and for sake of productivity, using paths you define for where software expects certain directories while improving organization can help to perform tasks more quickly, and make it stupid simple to back up everything for later use when the original media quits working.
That’s right — This isn’t a cop-out for backing up, but rather this is a way of getting stuff done faster. With all of your stuff in a single place without any system files attached you have no excuse — and you’d be an idiot — to not image whole partition to another device for safe keeping when the reaper takes your original copies of effects to the great recycle bin in the sky.
You should also back up your /home partition. If that fails for any reason, a quick restore later and access to your stuff would be returned. If the system happens to quit working, having a backup of that isn’t strictly necessary but preferred.
If you keep everything on a single disk, then you should still clone the entire media. If you’ve done everything correctly, then these provisions to assert more agency and responsibility over your data can help to save it against corruption of ext4 partitions due to disgraceful dismounts and the warring conflict between your open-source system and Microsoft Windows’ updates.
Special thanks
@nam1962 for highlighting the deep linking feature of Discourse which allows for a ToC.
@bogdancovaciu for making me a damned fool with pointing out how forgetful I was of how HTML anchors worked. The table works now because of it.
