Increase system partition

[slimply]

♫♫♪♩♪♩ "Always look on the bright side of life…"♫♫♪♩♪♩

After setting up and testing BACKUPS, that is.

… And now you’ve got your earworm for today.

<saunters off whistling ♫♪♩♫♪♩ >

[bedna]

Ok, here is the reality.

Yes, things like that CAN occur, but do they ever?

I have read what you guys type above, and also that that stuff is never needed because you would not end up in a situation like that UNLESS you have multiple disks.

I am one of those persons that if I see conflicting information, just test myself so I created a situation for testing this about 6 months ago.
I use this drive as “everything drive”, ie if I download something, this is where it goes. I create big img files to work with my rpi:s, this is where it goes.
I even intentionally did something that is not recommended on btrfs, download torrents to it, because it can create a situation where balance could be needed, because of all the tiny chunks that are being created when downloading.

The data exchange on this drive is massive.
Not an ssd, but that doesn’t matter.
NEVER balanced it.
Lets take a look of how much space I would “save” if I ran a balance.

$ sudo btrfs fi df /media/fil-fitt
Data, single: total=1.23TiB, used=1.22TiB
System, DUP: total=8.00MiB, used=160.00KiB
Metadata, DUP: total=2.25GiB, used=1.53GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

sudo btrfs fi usage /media/fil-fitt
Overall:
    Device size:                   2.47TiB
    Device allocated:              1.24TiB
    Device unallocated:            1.23TiB
    Device missing:                  0.00B
    Device slack:                    0.00B
    Used:                          1.23TiB
    Free (estimated):              1.24TiB      (min: 639.74GiB)
    Free (statfs, df):             1.24TiB
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)
    Multiple profiles:                  no

Data,single: Size:1.23TiB, Used:1.22TiB (99.26%)
   /dev/sdb3       1.23TiB

Metadata,DUP: Size:2.25GiB, Used:1.53GiB (67.81%)
   /dev/sdb3       4.50GiB

System,DUP: Size:8.00MiB, Used:160.00KiB (1.95%)
   /dev/sdb3      16.00MiB

Unallocated:
   /dev/sdb3       1.23TiB

99,26%…

Again, you guys do you, but imho, a balance is never needed on a single device filesystem.
But ask me again in a year, but if this is 6 months of heavy usage, I do not think it will be much different in any amount of time.

[andreas85]

@slimply

If your file system is btrfs, you may want to ‘balance’ it.

• Do not balance daily
• Balance won’t break your SSD

Good information about Btrfs can be found on the wiki

Btrfs

Especially about how to use balance after using more than 80% of your space at one point

Btrfs

[megavolt]

@bedna

Again, you guys do you, but imho, a balance is never needed on a single device filesystem.
But ask me again in a year, but if this is 6 months of heavy usage, I do not think it will be much different in any amount of time.

Well there is enough unallocated space:

@bedna

Device unallocated: 1.23TiB

There is no issue as long as you have enough unallocated space. But what would happen if almost everything is allocated and it cannot create a new chunk for writing?

@bedna

I create big img files to work with my rpi:s, this is where it goes.

@bedna

download torrents to it, because it can create a situation where balance could be needed, because of all the tiny chunks that are being created when downloading.

Anyway… your examples are not “heavy usage”, it is rather “average usage”. And downloading torrents have zero effects on chunks, only file fragmentation. What has an effect effect are subvolumes/snapshots, write a lot small files, delete them or overwrite them. Torrent chunks are not small files and count rather as extents on btrfs, since torrents downloads are usually pre-allocated and every “torrent chunk” fill the gap of the pre-allocated sparse file.

But I must admit, that this thread is going off-topic. Maybe it is better to move this to Member Hub .

[cscs]

Similarly this note on Arch wiki has been there for at least 5 years…
(meaning it was pertinent then, but even more so today)

Avoiding unnecessary access to slow storage drives is good for performance and also increasing lifetime of the devices, although on modern hardware the difference in life expectancy is usually negligible.

Note: A 32GB SSD with a mediocre 10x write amplification factor, a standard 10000 write/erase cycle, and 10GB of data written per day, would get an 8 years life expectancy. It gets better with bigger SSDs and modern controllers with less write amplification. See also this endurance experiment when considering whether any particular strategy to limit disk writes is actually needed.

https://wiki.archlinux.org/title/Improving_performance#Reduce_disk_reads/writes

[bedna]

@megavolt

There is no issue as long as you have enough unallocated space. But what would happen if almost everything is allocated and it cannot create a new chunk for writing?

I have filled that drive to the brink at times (making sure to leave minimum 10% free ofc). I treat this drive like trash intentionally. (I also makes backups with snapshots, so If I loose the drive, no big deal)

@megavolt

your examples are not “heavy usage”, it is rather “average usage”.

If you expected me to leave a report on every single thing I use the drive for, you misunderstand.
I just pulled a few things out of my head.
I ABUSE THIS DRIVE INTENTIONALLY!

What you talk about is “overfilling the drive”. If you read the documentation of btrfs it clearly tells you not to do that! THAT is the problem, do not do that!
You should NOT have to rebalance a single drive btrfs filesystem if you follow the instructions of how to use btrfs.

[megavolt]

@bedna

Running balance without filters will take a lot of time as it basically move data/metadata from the whole filesystem and needs to update all block pointers.
Balance — BTRFS documentation

The filters can be used to perform following actions:

• convert block group profiles (filter convert)
• make block group usage more compact (filter usage)
• perform actions only on a given device (filters devid, drange)

@bedna

You should NOT have to rebalance a single drive btrfs filesystem if you follow the instructions of how to use btrfs.

Roger. Write down the 10 commandments of the BTRFS gods. Let’s see how many follow them.

@bedna

What you talk about is “overfilling the drive”. If you read the documentation of btrfs it clearly tells you not to do that! THAT is the problem, do not do that!

But it happens and to reduce the risk, compact the chunks after time (after 1,2,3 months or 1 year), which depends on usage, as the documentation offer. That doesn’t mean:

YOU SHALL NOT FILL UP A BTRFS PARTITION.

Also not:

YOU SHALL NOT BALANCE A SINGLE BTRFS PARTITION.

It says:

The primary purpose of the balance feature is to spread block groups across all devices so they match constraints defined by the respective profiles.

And where is a primary purpose, there can be also a secondary purpose. And compacting chunks is a valid purpose on a single btrfs partition. Period.

[bedna]

@megavolt

YOU SHALL NOT BALANCE A SINGLE BTRFS PARTITION.

It says:

The primary purpose of the balance feature is to spread block groups across all devices so they match constraints defined by the respective profiles.

So “all devices” implies ONE hard-drive?

My wording is bad though, it should say “you shall not have to balance a single drive btrfs filesystem”.
If you have multiple filesystems on the drive though, it CAN create situations.

Look, you do you, and if you feel recommending ppl doing scheduled balances each week, its on you.
I strongly recommend against it.

[megavolt]

@bedna

So “all devices” implies ONE hard-drive?

Now it’s getting really childish. So again, child-friendly:

You have 1 disk on the table and say: “These are all my disks.”. Then you look at the table and see that you only have one hard drive. Does “all disks” have to be more than one?

@bedna

My wording is bad though, it should say “you shall not have to balance a single drive btrfs filesystem”.
If you have multiple filesystems on the drive though, it CAN create situations.

Look, this all about a compacting data chunks:

btrfs balance start -dusage=90 /

That means: compact every chunk/block which is less than 90% full. You can go even lower and compact everything which less than %5 full.

If there is one which 91% full, it will skip it, everything below 90% will be compacted, means: they will be filled up to 90% with other data if possible. Really, it doesn’t harm. It just keeps all chunks almost full. Also useful when shrinking a btrfs partition.

@bedna

Look, you do you, and if you feel recommending ppl doing scheduled balances each week, its on you.
I strongly recommend against it.

Who talks about “every week”? A scrub should be done every (1-2) week, not a balance.

Please stop spreading misinformation. If that is your recommendation, based on your experience, then say it like that, but I was responding, because you say that it is a fact.

[bedna]

@megavolt

Look, this all about a compacting data chunks:

It’s not, its about MOVING the chunks so they don’t spread out over several blocks.

@megavolt

Please stop spreading misinformation. If that is your recommendation, based on your experience, then say it like that, but I was responding, because you say that it is a fact.

I just showed you a practical example, but is seems you have trouble seeing that. Very much NOT disinformation. it seems you did not read the whole thread because…

@megavolt

Who talks about “every week”? A scrub should be done every (1-2) week, not a balance.

@slimply

but with the lifetimes of today’s ssds the average user will still have several decades of use with a weekly or monthly balance.

Witch is the comment that started this whole conversation.

Again, you do you. I’m leaving this conversation now.

[slimply]

@bedna

Witch is the comment that…

'K, its Halloween.

@cscs

Arch Wiki:
Note: A 32GB SSD with a mediocre 10x write amplification factor, a standard 10000 write/erase cycle, and 10GB of data written per day, would get an 8 years life expectancy. It gets better with bigger SSDs and modern controllers with less write amplification. See also this endurance experiment when considering whether any particular strategy to limit disk writes is actually needed.

The principle stands. Now let’s run the numbers with today’s SSDs and increased (tripled) usage. I’ll be conservative with a size of 500GB and a write amplification of 5.

Size: 500GB / 32GB = 15.625
Write amplification: 10 / 5 = 2
Usage: 10GB * 3 = 30GB

Based on (Write cycles * Capacity) / (SSD factor * Data Written per year), the expectancy grows from size alone to 8 years * 15.625 = 125 years. The halved write amplification doubles it to 250 years. 1/3 of that from tripled usage still gives us 83 years.

That is not realistic, but a strong indicator that we really need not to worry about SSDs failing from flash failure. More likely is that the controller chip or any other of the logic dies before 83 years are over.

Now the devil is in the detail, and real life seldom matches exactly a calculation. To quote from https://www.howtogeek.com/891272/ssd-myths-and-misconceptions-people-still-believe/ (recommended):
(TBW = TeraBytes Written)

When held up against real-world use for your average consumer, TWBs are astronomically high. Most 1TB drives have a TWB of 600TB. To hit that TWB value in 5 years, you’d need to write 328.8GB to disk every single day.

Realistically, your SSD will outlive your computer and perhaps several upgrades. Only a personal note, I’ve been using SSDs as my primary OS drives since 2011, and I have yet to have a drive die. Every SSD in my household lives multiple lives working its way from PC to PC and project to project.

The experiences from the early years of the SSD market have been very valuable then, but by now they have become an unquestioned lore that is outdated.

Related: To get an impression what to expect in the first years of an SSD life, Backblaze has some statistics (also for HDDs, search the site for ‘drive stats’):

https://www.backblaze.com/blog/ssd-drive-stats-mid-2022-review/

---

Moderators: The thread is derailed from the original question – can you split BTRFS and SSD discussions into separate threads? “BTRFS: To balance or not to balance” and “Life expectancy of current SSDs” or so.