What is the current recommendation for Linux filesystem?

I also wonder why, but this package is just made to add scripts used when doing update-grub command when you have BTRFS file system. But I agree this shouldn’t be installed by default in a non BTRFS installation. I also agree this is off-topic and shouldn’t be in this thread.

2 Likes

as recommended here in case you are allocating a new partition to use in-between windows and linux, exfat seems to be the best option with least fuss. however if you are using existing NTFS partitions with data in them to avoid the hassle you can keep them as NTFS.

while ntfsfix is no chkdsk, it does basic repair. what most people forgo is that ntfsfix also marks every partition it checks as “dirty” irrespective of state of partition, to be cleared by an actual check with chkdsk later. ntfs3 is extra sensitive to NTFS status and does not mount “dirty” partitions. however you can avoid this using “-d” switch of ntfsfix.

i’ve resorted to using ntfs3 driver (mostly for less resource usage) with udisk2 so my NTFS partitions are auto-mounted on demand. i keep ntfs-3g driver as a backup driver to manually mount when i run into issues mounting with ntfs3.

I guess you want to use this driver: GitHub - maharmstone/btrfs: WinBtrfs - an open-source btrfs driver for Windows

Well I used it and it works, but it shouldn’t be used for critical data and in case of failures, you have to use Linux anyway. I also saw complete crashes, when using Win10 with this driver.

1 Like

I think it is a bad idea to share a partition with this kind of custom driver. NTFS to me is the way to go for a share with Windows. NTFS is well supported now in kernel 5.15+ with the NTFS3 driver (which requires only proper mount options to work), and perfectly suitable to share data. Create the partition from Windows, and use it on both Windows and Linux without issue.

For ntfs-3g it is the same, while it is well tested and has less bugs :slight_smile: However I still expect bugs in NTFS3 and therefore ntfs-3g is just the default. This btrfs driver is not custom, but rather a port to ReactOS, which as been adopted to real windows platforms.

If I have to choose, I would prefer an ext4 or btrfs driver for windows and not the way around. Windows drivers for both filesystems work, but they are far from being rock stable. At least for game data I would use these drivers.

To me it is by definition a custom driver, from the link you posted:

WinBtrfs is a Windows driver for the next-generation Linux filesystem Btrfs. A reimplementation from scratch, it contains no code from the Linux kernel

Also you can note I didn’t recommend using ntfs-3g implementation I specifically talked about NTFS3, which indeed may have bugs, but I disagree on your position.

I wouldn’t work on Linux file systems from Windows, and I would trust more the NTFS3 driver by a corporation that made working on file systems for almost 30 years its core business, than a driver rewritten from scratch on a GitHub project by one man. At the end of the day you do what you want but I don’t see how that makes more sense “the other way around”.

exFAT is the better choice for sharing with all OS without complication and access limitation.

I wouldn’t use NTFS on Linux systems, either, if it were up to your argument. FAT12/16/32 and exFAT also not.

Ah yes, now comes the “expert argument”, which however has little weight. Just because a well-known company does it doesn’t automatically mean that they are better, especially since they mainly program software for Windows.

Anyway, if both Windows and Linux are to co-exist, then they must also fully support the respective file systems. Well, Paragon has a solution for this under Windows, of course: Linux File Systems for Windows | Paragon Software But, it is the same as ntfs-3g; it works in user space and not on the kernel space. I used and bought it, but it is not really stable and fast with lots of files and well, BTRFS is read-only. So this driver is not as good as it should be.

My bad and sorry for the confusion, I forgot to change the text and just copy and paste.

The 3. I’m going to change from NTFS to BTRFS as personal data, I will not keep dual boot anyumore. The number 4 that I removed from the list I will keep as back-up at NTFS, becasue in case I need, it can be read at windows too.

  1. HD sata BTRFS personal data files
  2. HD sata NTFS Back-up personal data files shared with windows (this drive is unplugged, I use it only when doing back-ups)

:rofl: Come on, little weight? They indeed are expert in this domain. They make it their business. They officially added their code to the Kernel. This is a strong point.

Nope, that is a weak argument. A strong point would that it is “Made in Germany” and therefore German Know-How matters :clown_face:

However… whenever such an “expert argument” comes, it doesn’t mean anything. What has meaning is, how good the products are.

Talking about Filesystem drivers:
On Windows: Stable and reliable, but not good as it should be.
On Linux: Could be far better, but works.

ntfs3 vs ntfs-3g

i’m happy paragon managed to adapt its NTFS driver such that it could finally be part of the linux kernel (after many delays). from my use i only see a marginal performance increase compared to ntfs-3g, however efficiency-wise ntfs3 is way better (no more CPU/RAM hogging, at all).

however not all is smooth sailing with ntfs3. the moment you try to default to ntfs3 (ntfs-3g still being the default driver out-of-the-box, atleast in my setup) you run into ambiguity (somewhat mitigated) and feel there is some work still to be done. ntfs3 is supported by udisks2, but how it is implemented leaves little more to be desired. from the way i figured, there are issues arisen by adopting ntfs3 mounting mechanism to be a drop-in replacement for fuse+ntfs-3g system (specifically mounting options). you cannot expect everything to work right overnight just because there is a new kernel space driver. i’m sure there are decisions being made and work being done which will getover these soon.

for ntfs-3g, it seems its less finicky than ntfs3. i’ve had several instances where i could mount same NTFS partitions with ntfs-3g but not with ntfs3. ntfs3 will quit mounting for the slightest partition issues whilst ntfs-3g would mount it with no issues. (probably why you should be wary of using ntfs-3g in the first place).

For the ones that might ready this thread, I found the video below with nice explanation about ZFS that I will try to summarize below so people can have better conclusion.

ZFS is not recommended to single drive because the data will probably corrupt at any time, even if the user didn’t realize, it needs more then one driver to better work, due to how the system menage redundancies, so this sentence by itself eliminates my intention to use it.

ZFS compared to BTRFS and other file system has more overhead penalties and this is the reason it uses more memory and probably is slower (it has more features), and regarding the configuration you choose for your system it can became even worse.

1 Like

I tried ArchLinux with ZFS in VM for the first time.
But I noticed that rollback of ZFS snapshot is inflexible.

You can not rollback an old snapshot if other snapshots are newer than this old snapshot.
But you have to delete all newer snapshots except the selected old one for rollback,
or there is a trick that you can clone ZFS filesystem, then delete the newer snapshots, then rollback the old snapshot:
https://stackoverflow.com/questions/40659016/zfs-rollback-snapshot-but-keep-newer-snapshots

I find BTRFS rollback is more flexible than ZFS.


No wonder , if ZFS is good stable for RAID levels, but its rollback is hard (Not full tree capability).
BTRFS is not stable in RAID level 5 or 6, but its rollback is flexible like full tree capability.

1 Like

I tested myself benchmark of zfs vs. btrfs in two same VM in my same hardware (a single SSD). Both filesystems are running in Linux Kernel 5.17

Copy (Read and Write) speed test with zfs (default compression lz4) in the real world. (I can not change lz4 to zstd because it crashes grub after reboot, but lz4 would be bit faster than zstd)

  • Copy speed test of ZFS (Compression LZ4)
❯ sudo rsync -ah --progress /home/test/Desktop/backup /opt/backup_copy
sending incremental file list
backup
          4,19G 100%  456,51MB/s    0:00:08 (xfr#1, to-chk=0/1)

  • ZFS, test again:

❯ sudo rsync -ah --progress /home/test/Desktop/backup /opt/backup_copy1
sending incremental file list
backup
          4,19G 100%  653,51MB/s    0:00:05 (xfr#1, to-chk=0/1)

  • Copy speed test of BTRFS (Compression zstd)
❯ sudo  rsync -ah --progress /home/test/Desktop/backup /opt/backup_copy                                                                                                                                                        
sending incremental file list
backup
          4.19G 100%  598.44MB/s    0:00:06 (xfr#1, to-chk=0/1)
  • BTRFS, test again:
❯ sudo rsync -ah --progress /home/test/Desktop/backup /opt/backup_copy1
sending incremental file list
backup
          4.19G 100%  844.77MB/s    0:00:04 (xfr#1, to-chk=0/1)

Benchmarking with FIO:

  • ZFS
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --bs=4k --iodepth=64 --readwrite=randrw --rwmixread=75 --size=4G --filename=testfile

test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.29
Starting 1 process
Jobs: 1 (f=1): [m(1)][97.1%][r=110MiB/s,w=36.3MiB/s][r=28.1k,w=9304 IOPS][eta 00m:02s]
test: (groupid=0, jobs=1): err= 0: pid=194869: Sun Apr 24 12:46:52 2022
  read: IOPS=11.8k, BW=46.2MiB/s (48.4MB/s)(3070MiB/66495msec)
   bw (  KiB/s): min=35328, max=120664, per=98.37%, avg=46506.10, stdev=13001.85, samples=132
   iops        : min= 8832, max=30166, avg=11626.46, stdev=3250.46, samples=132
  write: IOPS=3950, BW=15.4MiB/s (16.2MB/s)(1026MiB/66495msec); 0 zone resets
   bw (  KiB/s): min=12064, max=38680, per=98.39%, avg=15545.64, stdev=4259.67, samples=132
   iops        : min= 3016, max= 9670, avg=3886.36, stdev=1064.90, samples=132
  cpu          : usr=1.88%, sys=36.05%, ctx=116626, majf=0, minf=8
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=785920,262656,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=46.2MiB/s (48.4MB/s), 46.2MiB/s-46.2MiB/s (48.4MB/s-48.4MB/s), io=3070MiB (3219MB), run=66495-66495msec
  WRITE: bw=15.4MiB/s (16.2MB/s), 15.4MiB/s-15.4MiB/s (16.2MB/s-16.2MB/s), io=1026MiB (1076MB), run=66495-66495msec
  • BTRFS
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --bs=4k --iodepth=64 --readwrite=randrw --rwmixread=75 --size=4G --filename=testfile

test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.29
Starting 1 process
test: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=1): [m(1)][100.0%][r=139MiB/s,w=45.8MiB/s][r=35.7k,w=11.7k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=24711: Sun Apr 24 12:40:16 2022
  read: IOPS=34.7k, BW=135MiB/s (142MB/s)(3070MiB/22660msec)
   bw (  KiB/s): min=129624, max=149544, per=100.00%, avg=138802.58, stdev=4058.99, samples=45
   iops        : min=32406, max=37386, avg=34700.60, stdev=1014.75, samples=45
  write: IOPS=11.6k, BW=45.3MiB/s (47.5MB/s)(1026MiB/22660msec); 0 zone resets
   bw (  KiB/s): min=42672, max=49464, per=100.00%, avg=46384.73, stdev=1385.39, samples=45
   iops        : min=10668, max=12366, avg=11596.16, stdev=346.36, samples=45
  cpu          : usr=3.85%, sys=87.35%, ctx=3168, majf=0, minf=6
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=785920,262656,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=135MiB/s (142MB/s), 135MiB/s-135MiB/s (142MB/s-142MB/s), io=3070MiB (3219MB), run=22660-22660msec
  WRITE: bw=45.3MiB/s (47.5MB/s), 45.3MiB/s-45.3MiB/s (47.5MB/s-47.5MB/s), io=1026MiB (1076MB), run=22660-22660msec
2 Likes

ZFS is for integrity, reliability, records-based replications, and long-term storage. It’s not meant for raw performance. I have no idea where the idea came from that it’s “faster” and better for gaming?

It also requires more of your system, due to everything it does under the hood.

Inline compression. Inline checksums. Redundant checksum data for a single record of user data. The freaking ARC (it’s a beast, and wants to use up as much RAM as is feasible.) And much more.

:question: Don’t know? Can’t decide? Windows not involved?

  • EXT4
    – Good for home, root, data, and external partitions. It works. It’s tried-and-true.

:question: Want to exploit SSD / NVMe performance? Feeling a bit “on the edge”?

  • F2FS
    – Best kept to root and things like var or opt, maybe not for home (use EXT4 instead)

:question: Got a beast system with a powerful CPU, many cores/threads, and much RAM? Dealing with large files and multimedia more than the typical user? Windows not involved?

  • XFS
    – Good for home, root, data, and external partitions.

:question: Got a grip on filesystems? Want to leverage snapshots, rollbacks, and inline compression? Up to acquiring extra knowledge and learning? Do you understand the concept of subvolumes? Windows not involved?

  • BTRFS
    – Probably best left to root, var, opt, and so on. Maybe keep home as EXT4.

:question: Need something shared between Linux and Windows?

  • exFAT
    – Nothing fancy. Works on Windows and Linux. Fine for internal or external drives. Only if you really need OS-interoperability. Make sure to use exfatprogs (Samsung), rather than exfat-utils (FUSE).

I left out ZFS for the reason that it’s best reserved for non-desktop NAS storage; and because of the licensing issues, in which its maintenance and development is outside of mainline kernel development. This can (and has) introduced bugs and regressions with kernel hops.


EDIT: The above is a quick “back of a napkin” overview if you can’t decide. It’s not comprehensive, nor is it a “hard recommendation”. It’s really just a “I don’t want to look into this too deeply. Give me the bird’s-eye view.”

2 Likes

Hello, to know, you need to ready the posts, otherwise …

In the first post there is a video from a youtuber that made a filesystem benchmark, and my questions was based on the data he shown, he said that in his tests ZFS was the fastest one, and sometimes the EXT4. I’m not able to make the tests myself to check if the data was wrong or right, so I asked here in the forum.

ZFS is very suitable for enterprises or mainframes with databases on server when you need secure data. RAID level e.g. 5 or 6 too, FreeBSD uses ZFS by default.

I never guessed where this error comes from.
Software & system are designed to use swap.
Blocking it will make them behave a way they are not designed to behave.

~]$ free
               total       utilisé      libre     partagé tamp/cache   disponible
Mem:           15397        8501        1468         474        5426        6096
Partition d'échange:      16383           1       16382