Why stable ISO images has a lot greater size than unstable images although all was built on same day?

alven · 19 November 2021 20:49

I compared daily unstable (Release 202111190248 · manjaro-plasma/download · GitHub) and stable ([Stable Update] 2021-11-19 - Kernels, Gnome 41.1, Plasma 5.23.3, Frameworks 5.88, LxQt 1.0, Xorg-Server 21.1, Mesa) images.

The package list comparison results in a bit ahead of unsable package version, not many about 20 packages only. But all version set by names are the same. So slightly difference in image sizes are expected.

du * --bytes --human-readable
3.4G    manjaro-kde-21.11-development-unstable-211119-linux515.iso
2.7G    manjaro-kde-21.11-development-unstable-minimal-211119-linux515.iso

du * --bytes --human-readable 
3.7G    manjaro-kde-21.2pre1-211119-linux515.iso
3.0G    manjaro-kde-21.2pre1-minimal-211119-linux515.iso

or the same but bytes-scaled:

du * --bytes
3575742464      manjaro-kde-21.11-development-unstable-211119-linux515.iso
2873233408      manjaro-kde-21.11-development-unstable-minimal-211119-linux515.iso

du * --bytes
3885817856      manjaro-kde-21.2pre1-211119-linux515.iso
3183362048      manjaro-kde-21.2pre1-minimal-211119-linux515.iso

Slightly. But not so huge like that.

May be some compression was not applied or applied another method or why?

Are the stable images OK?

alven · 20 November 2021 09:12

Today we got new daily unstable ISO updates:

du * --bytes --human-readable
3.6G    manjaro-kde-21.11-development-unstable-211120-linux515.iso
3.0G    manjaro-kde-21.11-development-unstable-minimal-211120-linux515.iso

or bytes-presented

du * --bytes                 
3863400448      manjaro-kde-21.11-development-unstable-211120-linux515.iso
3160891392      manjaro-kde-21.11-development-unstable-minimal-211120-linux515.iso

Now I see almost no size difference between today’s unstable and yesterdays’s stable ISO images.
unstable ISO “inherits” that +300 MB from it. But what exactly became bigger and why?

What package got changes if to compare unstable from yesterday and unstable from today:

cryptsetup        2.4.1-3   -> 2.4.2-1
lib32-libdrm      2.4.107-1 -> 2.4.108-1
lib32-libelf      0.185-1   -> 0.186-1
lib32-libtasn1    4.17.0-1  -> 4.18.0-1
libelf            0.185-1   -> 0.186-1
linux515          5.15.2-2  -> 5.15.3-1
mhwd-nvidia       495.44-4  -> 495.44-5
mhwd-nvidia-470xx 470.86-2  -> 470.86-3
zensu             0.3-2     -> 0.3-3

How much size does the packages have?

~/Desktop ❯ pacman-mirrors -G
unstable

~/Desktop ❯ sudo pacman -Syyu 
:: Synchronising package databases...
 core                                                                                 170.0 KiB   315 KiB/s 00:01 [####################################################################] 100%
 extra                                                                               1928.6 KiB  1776 KiB/s 00:01 [####################################################################] 100%
 community                                                                              6.8 MiB  2.61 MiB/s 00:03 [####################################################################] 100%
 multilib                                                                             175.7 KiB   296 KiB/s 00:01 [####################################################################] 100%
:: Starting full system upgrade...
 there is nothing to do

~/Desktop ❯ pacman -Si cryptsetup lib32-libdrm lib32-libelf lib32-libtasn1 libelf linux515 mhwd-nvidia mhwd-nvidia-470xx zensu | grep -i "repo"          
Repository      : core
Repository      : multilib
Repository      : multilib
Repository      : multilib
Repository      : core
Repository      : core
Repository      : extra
Repository      : extra
Repository      : community

~/Desktop ❯ pacman -Si cryptsetup lib32-libdrm lib32-libelf lib32-libtasn1 libelf linux515 mhwd-nvidia mhwd-nvidia-470xx zensu | grep -i "download size"
Download Size   : 590.81 KiB
Download Size   : 149.43 KiB
Download Size   : 477.43 KiB
Download Size   : 39.01 KiB
Download Size   : 473.10 KiB
Download Size   : 99.58 MiB
Download Size   : 22.89 KiB
Download Size   : 23.05 KiB
Download Size   : 23.72 KiB
~/Desktop ❯

So even if all those 9 packages would never included and now was included, than their total size is less than 105 MiB.

But where that +300 MiB ISO size increase came from?
Likely ISO format change (may be compression method).

So the questions are:

what was changed and why?
is it OK to have +300 MB with almost the same package versions and no any single additional package?

omano · 20 November 2021 09:48

Compare the ISO profiles that may answer your question? Also a package size is not the installed size.

Strit · 20 November 2021 10:10

Well, some packages might not be installed in the dev ISO, but will be on the stable one.

alven · 20 November 2021 10:42

Yeah, previously I also though that, but I compared packages list:
typical stable with typical unstable,
minimal stable with minimal unstable.
Comparison result goes like this:

“set” I meant list of packages - packages set.
So it is not the case.
I did not mention that today, but I stays in context of initial post: if to compare yesterday’s unstable to today’s unstable then I see exactly the same set of packages, no one was removed or added. Only those 9 packages changed their versions.
I did that comparison by *.pkg files comparing, which located in the same post with ISO images.

Dan, if you daily build ARM images, then recent (from yesterday evening or today) ARM ISO image versions did not get that comparable size increase? I wonder if it is some global settings involved, under which ISO imaged of all architectures builds (if such settings exists).

Still have queued to do omano’s suggestion.

linux-aarhus · 20 November 2021 10:51

Now that is a fact - some packages is 1/10 of the installed size - zstd compression at work.

Sometimes you get an update which installs more than 3GB but is only 800MB in the download queue

alven · 20 November 2021 11:09

I knew that:

~/Desktop ❯ pacman -Si linux-firmware | grep -i size
Download Size   : 183.40 MiB
Installed Size  : 731.19 MiB

~/Desktop ❯ pacman -Si libreoffice-still | grep Size 
Download Size   : 135.70 MiB
Installed Size  : 413.87 MiB

I thought that packages inside of ISOs are the same as mirror servers has: they are not pre-installed inside of image (only some core system packages installed to start to run the image), but while ISO starts that packages got unpacked, installed into RAM and run.

linux-aarhus · 20 November 2021 12:43

The size of the ISO is depending on the number of packages installed with the system.

Another factor is if the ISO has been enabled for office installation - usually a choice between libreoffice and freeoffice.

That alone set a large difference on the ISO size because both packages is packaged in the mhwd overlay.

freeoffice ~150MB
libreoffice ~150MB

➜  ~ ls -l /data/repos/manjaro/pool/overlay/freeoffice*.pkg.tar.zst 
-rwxr-xr-x 1 root root 152904274 Oct 19 19:08 /data/repos/manjaro/pool/overlay/freeoffice-2021.1036-2-x86_64.pkg.tar.zst
➜  ~ ls -l /data/repos/manjaro/pool/sync/libreoffice-(fresh|still)-7.*.pkg.tar.zst
-rwxr-xr-x 1 root root 148379495 Nov 19 00:46 /data/repos/manjaro/pool/sync/libreoffice-fresh-7.2.2-3-x86_64.pkg.tar.zst
-rwxr-xr-x 1 root root 142287524 Nov 19 00:46 /data/repos/manjaro/pool/sync/libreoffice-still-7.1.7-2-x86_64.pkg.tar.zst

alven · 20 November 2021 16:28

The only fartor I found is the [mhwd] add 470xx · manjaro-plasma/iso-profiles@1cc952d · GitHub

But in that linked commit what is the file format/markup?

>nonfree_multilib lib32-nvidia-utils
opencl-mesa

I mean what the start > brings?
I did not find that info in the iso-profiles/README.md at sway · manjaro-sway/iso-profiles · GitHub
If they are became preinstalled packages, than does that 3 new packages consumes +300 MiB of ISO image?

Still did not find the answer:

all those which was added in that commit are:

linux515-nvidia-470xx
nvidia-470xx-utils
lib32-nvidia-470xx-utils

If to check them:

~/Desktop ❯ pacman -Si linux515-nvidia-470xx nvidia-470xx-utils lib32-nvidia-470xx-utils | grep -i repo
Repository      : extra
Repository      : extra
Repository      : multilib

~/Desktop ❯ pacman -Si linux515-nvidia-470xx nvidia-470xx-utils lib32-nvidia-470xx-utils | grep -i "installed size"
Installed Size  : 43.37 MiB
Installed Size  : 423.97 MiB
Installed Size  : 115.86 MiB

~/Desktop ❯ pacman -Si linux515-nvidia-470xx nvidia-470xx-utils lib32-nvidia-470xx-utils | grep -i "download size"     
Download Size   : 43.15 MiB
Download Size   : 191.84 MiB
Download Size   : 39.46 MiB
~/Desktop ❯

So, installed size: 580 MiB
Download size: 275 MiB

But all the packages already was inside of earlier lighter ISOs, so download size of 275 MiB could not cover that additional ~300MiB.
If they was pre-installed inside of ISO: switch from next version gave + 300 MiB? So their size now 580 MiB and a day before was 580 - 300 = 280 MiB, so the size increased twice? Can’t believe that.

Still do not know why and what happen.

omano · 20 November 2021 23:34

it’s for Full ISO, not for Minimal.