I updated yesterday. It went through fine. My system is up and running. Problem is with my external usb drive. It has 3 ntfs partitions. Two of them mount fine. But one doesn’t. There is some mention of Dirty, and failing to load $BadClus file in journalctl report. Prior to the update, it was ntfs-3g mounting them. Now it is ntfs3. It mounts successfully now when I run mount command on terminal with -t ntfs-3g. I saw discussion in this forum on this ntfs driver issue a year ago. Don’t know why I faced this problem after the recent update.
I have already run chkdsk deep scan with /r option and it didn’t report any errors. But I didn’t use /b option so that $BadClus file would get regenerated. It takes some 8 hours to scan. Later tonight.
I bet your fstab file has auto as the mount option instead of ntfs-3g.
I’m speculating but I assume the logic that selects the default ntfs driver has recently changed. It used to be that if auto mounted or type=ntfs was used it would look for the ntfs-3g driver and if it wasn’t installed it would fallback to the kernel driver. I feel it now selects the kernel driver over the ntfs-3g driver. I can’t find any documentation to suggest this is the case but from the reports I’ve read, everyone that has had a problem has been auto-mounting and the driver they’re using has changed.
Your post(s) have been moved to a dedicated topic.
A positive indicator of bad sectors/clusters. These can grow over time if not regularly attended to with chkdsk.
ntfs-3g has no facility to protect the volume from ongoing damage – there is no knowing the extent of damage, when, or how it may have occurred (typically, damage accrues over time) – ntfs-g3 keeps on mounting the volume regardless of the dirty bit being set, until the damage is such that it can no longer.
The timing is likely irrelevant.
Unlike ntfs-3g, ntfs3does warn you that the dirty bit is set (and that a volume is likely damaged) – and it prevents the volume from mounting; the error message warns you and you are expected to fix it, if possible (in the case of bad sectors, chkdsk will mark them as bad so that data is not written to those areas in future).
The reason for this is already explained – ntfs-3g will mount a volume even if it’s damaged – this does, however, make ntfs-3g handy for copying retrievable data to another location in the event of damage to a volume.
That’s a conservative estimate.
For the benefit of passers-by; once an advancedchkdsk scan is initiated it should never be interrupted – regardless of how long it might take.
All things being equal, it’s not really a problem; however, the default has been ntfs3 for a significant amount of time; this suggests that you blacklistedntfs3 at some point in favour of ntfs-3g. The reversal of that blacklisting remains a mystery.
Yes, the kernel module (ntfs3) is the default driver to handle ntfs – this change occurred only a few years ago, as ntfs3 was considered a better option – although the ntfs-3g camp offered resistance to the idea for some time; and still do, on the odd occasion.
Much of the resistance was attributed to not understanding (or refusal to understand) why a filesystem driver should actively prevent mounting a volume when damaged.
Soon, it will be even more fun Welcome to the new NTFS (without 3 anywhere in the name) in kernel module from kernel 7.1 onwards. There will be 3 different versions, each with its perks. A vast choice between different bugs, everyone can choose his favourite:)
I’m not sure that’s entirely true. From what I’ve seen yes the ntfs3 driver has been in the kernel and it will get used if there is no other option, but if the ntfs-3g driver is installed it will preferentially use that instead - automatically. Until recently, where it now seems to preference the kernel driver - automatically regardless of whether the ntfs-3g driver is installed.
Which is why people are now suddenly running into issues when their fstab is set to auto or ntfs.
The kernel module should first be blacklisted and then the userland driver will automatically take precedence.
Example; one can blacklist ntfs3 simply by adding a drop-in file to /etc/modprobe.d/ with the content: blacklist ntfs3
The file can be named anything descriptive enough to remember why it’s there; example:
/etc/modprobe.d/blacklist-ntfs3.conf
If ntfs-3g is installed, it should assume the role.
The decision to blacklist ntfs3 can be reversed by deleting the drop-in file.
Under normal circumstances ntfs3 and ntfs-3g can co-exist on a system – it is sometimes suggested to keep ntfs-3g around for casual uses – for manual mounting of a disk or removable USB, for example.
It’s only when the preference is to use ntfs-3g globally that blacklisting the ntfs3 kernel module is suggested.
History has shown, however, that many jumped on the concept of blacklisting ntfs3 due to their lack of understanding of how ntfs3 worked.
Several even went as far as to blamentfs3 for corrupting their ntfs volume, whereas all it did was to inform them of damage, and prevent it from being mounted and being damaged further.
Except OP’s question included a description of what is occurring, and:
And there are numerous other people reporting similar things with the last update.
The util-linux package was updated in the last update, and it includes a change to how it prioritises ntfs drivers that affects anyone that has their mount options set to auto.
This is the problem OP describes.
And the issue is Arch and Manjaro (and many other distributions) have for a very long time setup fstab with auto for NTFS drives which used the ntfs-3g driver if it was installed and configured correctly BEFORE falling back to ntfs3. But that has now changed to preference the ntfs3 driver so a bunch of people are now running into problems because of the differences in how each driver handles errors and dirty bits.
The solution is for users to decide on which driver best suits their needs and configure fstab to use that driver specifically. (The linux kernel documentation suggests for stability and general use the ntfs-3g driver continue to be used). And also to correct the errors on the ntfs drive regardless of which driver they choose.
and:
This is outdated information particularly when a known hierarchy is built into util-linux and when that same package provides a way to specify which driver to use. But it is also irrelevant if OPs fstab uses the auto mount option which is the more likely scenario since most distros (Manjaro included) use auto by default.
With the disclaimer that I don’t have any ntfs volumes and that I don’t use Microsoft Windows, but adding to this that I do know a thing or two about the kernel, here’s how I have always known it to work…
/etc/fstab is parsed by /usr/bin/mount, or /bin/mount in legacy installations which haven’t done the /usr merge yet.
If the filesystem type field in /etc/fstab or the mount command at the prompt specifies the keyword auto — or no filesystem type is specified in the mount command at the prompt — then the kernel will make an educated guess regarding what driver to use.
For this, it will always attempt to use drivers which have either been hard-linked into the kernel binary or loaded into the runtime kernel as modules. And here’s the rub: ntfs-3g is a userspace driver — it is loaded into the FUSE framework, not into the kernel. As such, it can never take precedence over an in-kernel driver for the same filesystem type — in this case being ntfs —unless…
the pertinent in-kernel driver is blacklisted; or
/etc/fstabexplicitly calls upon ntfs-3g for mounting an ntfs filesystem.
Now, if the ntfs filesystem is mounted through other means — e.g. through a mount-on-demand operation initiated by a file manager with udisks2 at the back-end — then things might be different, because then an additional configuration might be in place through a config file which specifies that the FUSE driver should be preferred over the in-kernel driver.
Without such a trigger from a usserland application such as a file manager, the kernel heuristics as detailed in the paragraphs above apply, and then the only way to have the kernel use ntfs-3g instead of ntfs3 (or however it is called these days) is to blacklist the in-kernel driver.
With all due respect, that is what I would have assumed too, but reading through the merge that made this change that information is incorrect.
I highly recommend anyone that wants to understand this process to go read the links I included above. The discussion on these are by the authors and they contradict what has been written here.
The information in the links you posted does however specify that the behavior is a kernel compile-time option. So then it depends on how the kernel was compiled, and in Manjaro, @philm is the developer who builds the kernels.
As I don’t need or use ntfs, I also have no dog in this particular fight, and thus I will take my leave from this conversation.
I’m closing this thread because the original question has been answered, and the discussion was in danger of derailing from the factual solution.
The Final Answer:
The user’s problem was caused by an update to util-linux (version 2.42). This update changed the system’s behavior to prefer the in-kernel ntfs3 driver for mounting NTFS drives over the userspace ntfs-3g driver when fstab uses the auto option.
Because the kernel driver is stricter, it refused to mount a volume with a “dirty” bit set. The solution for anyone encountering this is to either:
Run chkdsk on the drive from Windows to clear the dirty bit.
Or, change the mount type in /etc/fstab from auto to ntfs-3g to force the system to use the older driver.
Welcome to the new NTFS (without 3 anywhere in the name) in kernel module from kernel 7.1 onwards. There will be 3 different versions, each with its perks. A vast choice between different bugs, everyone can choose his favourite:)