Baloo File Indexing and BTRFS

I am currently on KDE, EXT4. Currently considering converting to Gnome.
I tried BTRFS (especially for /home) but Baloo keeps reindexing all files with each and every snapshot, thus makes the system unusable. It is a bug filed with Baloo. (after 2 snapshots searching for a unique file gives 3 results pointing to the same file)

I wonder if Gnome + BTRFS uses something other than Baloo for file content indexing and file content search that does not have this bug?

Does it have the same issue of keeping reindexing thousands of files with every snapshot? Multiple results for the same file when search? Ever growing index?

If such bugs are not present in Gnome I will surely convert to Gnome + BTRFS.

GNOME uses tracker3 by default, it can search file name, but it can not search content in file.

Baloo can search file name and content in files (it supports some formats) and nonexistent files (like deleted files) in Baloo cache, but you clear the cache manually, that is why it is awkward.

I am using kfind in Dolphin by default that can search file name and content in files too.

KFind is fast enough if you use NVME SSD

Thanks a lot @Zesko
Does “kfind” work ok with BTRFS? No continuous indexing for each snapshot? No multiple search results?
UPDATE: Just tried “kfind”, it does not seem to be indexed search!

It works fine with BTRFS and does not index for each snapshot unlike Baloo.

I just tried it… it is too slow for me. I have thousands of books and research papers.

Do you know if Gnome file search can be ok? Indexed without the Baloo bug? Is there any indexed search within the file manager in Gnome?

KFind does not use cache. That is why it is slower than Baloo cache (but old deleted files inside).

Gnome tracker3 uses its own cache too. The indexing is stored in ~/.cache/tracker3/files/
I think it is fast like Baloo.

So, does it have the same Baloo bug with BTRFS (continuous indexing every snapshot)?

Yeah, it works fine with BTRFS without reindexing for each snapshot

1 Like

WOW! That’s great news!
I will try reinstall Manjaro with Gnome. Hopefully I can keep my data, I’ll backup anyway before I install.

I am thinking of BTRFS just for better security.
What you think? Or better stay with my current system? (KDE, EXT4, Time Shift). I should be OK with Time Shift, I can anyway boot from live USB and while on live session I can restore from the already saved Time Shift snapshot?
Is it worth it to convert to Gnome + BTRFS?

Do not forget. Gnome tracker3 can not search content of file (PDF), but Baloo can.
(Maybe I missed new configuration of tracker3 to enable searching content of file)

When in doubt, you do not use conversion, just do backup then reinstall then copy your data back.

1 Like

I see… I got it.
So, I will stick with what I have for now, hopefully Baloo gets fixed!
Thank you for your help @Zesko

Off topic

If you want to use Baloo search for several thousand books and BTRFS for /home and / in the partition, try to do the better strategy:

  • Create a partition with BTRFS for /home and /.
  • Create a new partition with EXT4 only for several thousand books in a new directory /books that doesn’t need snapshot.
  • Edit /etc/fstab to mount this partition EXT4.
  • KDE Menu → File search (Baloo) → Create a new configuration → Select only this directory /books
    See the example like:

OK. I will try it.
I will appreciate any guidance if I can “convert” to BTRFS without reinstalling. If not… I will try again.

I tried the config you mentioned befor but had trouble mounting the EXT4, I didn’t
know of:

EDIT: This is my /etc/fstab

# /etc/fstab: static file system information.
# Use 'blkid' to print the universally unique identifier for a device; this may
# be used with UUID= as a more robust way to name devices that works even if
# disks are added and removed. See fstab(5).
# <file system>             <mount point>  <type>  <options>  <dump>  <pass>
UUID=BF63-1066                            /boot/efi      vfat    umask=0077 0 2
UUID=2d2a4da8-87e3-426c-81e0-4e034745e2ee /              ext4    defaults,noatime 0 1
tmpfs                                     /tmp           tmpfs   defaults,noatime,mode=1777 0 0

I don’t see where to or how to add paths to EXT4? Sorry for being that illiterate.

EDIT: I thought of something like what you suggested BuildIso vs. BTRFS System Backup/Restore - #17 by Zesko but had a problem of handling the EXT4 partition!
Thanks a lot @Zesko
Amazing @Zesko and community as usual.

Oh, do you mean EXT4 has the same issue that it reindexes files after reboot again?

This seems like excellent advice. I love btrfs, but it isn’t suitable for every use case (for example, virtual machines), so mixing it with ext4 or xfs can let you have nest of the both worlds.

To be honest, you might not want to have btrfs for your /home partition either. At least I find myself only snapshotting the / and purposefully excluding /home so I can rollback updates without losing changes in my real data. In theory you could do all sorts of neat stuff at /home too, but I never end up using those features.

1 Like

No. I installed BTRFS and created an EXT4 Partition.
I found the BRFS partition had /home
The EXT4 partition was not mounting, was like a partition just seen but not accessible.

Maybe it needed to be included in “/etc/fstab” as you mentioned to get mounted automatically, then added to Baloo to index.

(unfortunately I dont know how to:

  • include EXT4 partitoin in “/etc/fstab”
  • add the EXT4 partition (or a folder there and subfolders) to Baloo.)

Sorry, I am taking much of your time.

Sounds great :+1: :+1: :+1: @Chrysostomus
All BTRFS and EXCLUDING /home from snapshots!
I hope this will not cause the Baloo bug of reindexing all the time and having multiple entries for the same file!
This can be the solution to the Baloo bug.
But… I don’t know how to exclude /home from being snapshot!
Is it done through snapper? or other way?

Sorry I have no big experience with BTRFS.

There is partition management in Manjaro Calamares installer, you can use it to create your own partitions manually. It will generate /etc/fstab automatically.
You don’t need to configure /etc/fstab manually.

Make a backup first in an external hard drive, then reinstall Manjaro.

If you have a 1 TB hard drive in your computer that supports UEFI.

  • Boot partition: 300 MB with FAT32 for /boot/efi
  • System partition: 100 - 150 GB with BTRFS for /
  • home partition: 750+ GB with EXT4 for /home
  • Swap partition: ??? GB with Swap depends on RAM size (If 16 GB RAM , then 16 GB Swap)

Manjaro btrfs setup does this by default by placing /home on subvolume @home. Subvolumes are not recursive (and we don’t use nested subvomues anyway), so when you snapshot @, @home is left untouched.

This is enough to exclude /home from snapshots, but it doesn’t give btrfs the same performance profile as other simpler filesystems. So if you use virtual machines or databases that are housed on a btrfs volume, you need to stuff like disable copy on wrote on those files in order not to get massive performance penalty.

1 Like

It is impossible for full EXT4 system. @limotux is using full EXT4 in just one partition.
Look at his FSTAB:

That is why he should reinstall Manjaro using BTRFS for /and EXT4 for home, when he want to use BTRFS for creating snapshots and Baloo search for home.

1 Like