@cscs
Only kernel-modules-hook
Something does seem wonky.
I also have kernel-modules-hook
installed, and usr/lib/modules
has old kernel modules fully populated - 123 directories, 675 files
I uninstalled 5.15 not that long ago, yet itβs still listed here, and I see modules for seemingly every version of 6.10 released, but only one for 6.6 and only one for 6.11 (6.11.0-7 which I just reinstalled today), I did have 6.11.0-3 but uninstalled that.
/usr/lib/modules/
βββ 5.15.163-2-MANJARO
βββ 6.10.0-1-MANJARO
βββ 6.10.0-2-MANJARO
βββ 6.10.0-3-MANJARO
βββ 6.10.0-4-MANJARO
βββ 6.10.10-2-MANJARO
βββ 6.10.10-3-MANJARO
βββ 6.10.11-1-MANJARO
βββ 6.10.11-2-MANJARO
βββ 6.10.11-3-MANJARO
βββ 6.10.1-1-MANJARO
βββ 6.10.3-1-MANJARO
βββ 6.10.3-2-MANJARO
βββ 6.10.3-3-MANJARO
βββ 6.10.3-4-MANJARO
βββ 6.10.3-5-MANJARO
βββ 6.10.3-6-MANJARO
βββ 6.10.4-1-MANJARO
βββ 6.10.5-1-MANJARO
βββ 6.10.5-2-MANJARO
βββ 6.10.5-4-MANJARO
βββ 6.10.6-10-MANJARO
βββ 6.10.6-11-MANJARO
βββ 6.10.6-12-MANJARO
βββ 6.10.6-1-MANJARO
βββ 6.10.6-2-MANJARO
βββ 6.10.6-3-MANJARO
βββ 6.10.6-7-MANJARO
βββ 6.10.6-8-MANJARO
βββ 6.10.7-1-MANJARO
βββ 6.10.7-2-MANJARO
βββ 6.10.7-3-MANJARO
βββ 6.10.7-4-MANJARO
βββ 6.10.8-1-MANJARO
βββ 6.10.8-3-MANJARO
βββ 6.10.8-4-MANJARO
βββ 6.10.9-1-MANJARO
βββ 6.10.9-2-MANJARO
βββ 6.11.0-7-MANJARO
βββ 6.6.52-1-MANJARO
βββ extramodules-5.15-MANJARO
βββ extramodules-6.6-MANJARO
Aha.
I think I found it. In my case at least.
And anyone else who has this effect while using kernel-modules-hook
.
Have you started or enabled linux-modules-cleanup.service
?
Give it a go and then review your directories again.
(if you use something like systemctl start linux-modules-cleanup
you may need to wait a while for it to complete β¦ especially if you have many old modules directories)
Users may need to install the kernel-modules-hook
package to access that service:
pamac list -f kernel-modules-hook ξ² β
/usr/lib/systemd/system/linux-modules-cleanup.service
/usr/lib/tmpfiles.d/linux-modules-cleanup.conf
/usr/share/libalpm/hooks/10-linux-modules-post.hook
/usr/share/libalpm/hooks/10-linux-modules-pre.hook
/usr/share/licenses/kernel-modules-hook/UNLICENSE
pamac info kernel-modules-hook ξ² β
Name : kernel-modules-hook
Version : 0.1.7-3
Description : Keeps your system fully functional after a kernel upgrade
URL : https://github.com/saber-nyan/kernel-modules-hook
Licenses : UNLICENSE
Repository : extra
Installed Size : 2.4 kB
Groups : --
Depends On : rsync
Optional Dependencies : --
Provides : --
Replaces : --
Conflicts With : --
Packager : T.J. Townsend <blakkheim@archlinux.org>
Build Date : Sat 13 Jul 2024 05:06:11
Validated By : MD5 Sum SHA-256 Sum Signature
It isnβt part of the default installation (at least it wasnβt when I set up my new PC a year ago), and is aimed at users who do not immediately reboot after a kernel update.
Of course.
That is the subject.
It is a remedy for in the case of using it and therefor experiencing the old modules folder.
(added extra bold)
Rechecking the progress of this thread, I got curious β¦
I always reboot my system after maintenance tasks involving the kernel.
I remember all the issues surrounding the - at the time - hot topic that prompted for inclusion of the hooks which should keep the kernel alive by holding kernel modules until reboot.
07:33:53 β [fh@tiger] ~
$ mhwd-kernel -li
Currently running: 6.11.0-7-MANJARO (linux611)
The following kernels are installed in your system:
* linux611
07:34:00 β [fh@tiger] ~
$ tree /usr/lib/modules -L2
/usr/lib/modules
βββ 6.10.5-1-MANJARO
β βββ modules.weakdep
βββ 6.11.0-7-MANJARO
βββ build
βββ kernel
βββ kernelbase
βββ modules.alias
βββ modules.alias.bin
βββ modules.builtin
βββ modules.builtin.alias.bin
βββ modules.builtin.bin
βββ modules.builtin.modinfo
βββ modules.dep
βββ modules.dep.bin
βββ modules.devname
βββ modules.order
βββ modules.softdep
βββ modules.symbols
βββ modules.symbols.bin
βββ modules.weakdep
βββ pkgbase
βββ updates
βββ vmlinuz
6 directories, 18 files
07:35:19 β [fh@tiger] ~
$ pamac search kernel-modules-hook --no-aur
kernel-modules-hook 0.1.7-3 extra
Keeps your system fully functional after a kernel upgrade
07:35:34 β [fh@tiger] ~
$ pamac search -f modules.weakdep --no-aur
/usr/lib/modules/6.1.110-1-MANJARO/modules.weakdep is owned by linux61
/usr/lib/modules/6.10.10-2-MANJARO/modules.weakdep is owned by linux610
/usr/lib/modules/6.6.49-2-rt41-MANJARO/modules.weakdep is owned by linux66-rt
/usr/lib/modules/6.6.51-1-MANJARO/modules.weakdep is owned by linux66
/usr/lib/modules/6.10.2-5-rt14-MANJARO/modules.weakdep is owned by linux610-rt
/usr/lib/modules/5.4.284-1-MANJARO/modules.weakdep is owned by linux54
/usr/lib/modules/5.10.226-1-MANJARO/modules.weakdep is owned by linux510
/usr/lib/modules/4.19.322-1-MANJARO/modules.weakdep is owned by linux419
/usr/lib/modules/5.15.167-1-MANJARO/modules.weakdep is owned by linux515
/usr/lib/modules/6.1.108-2-rt40-MANJARO/modules.weakdep is owned by linux61-rt
/usr/lib/modules/6.11.0-rc7-3-MANJARO/modules.weakdep is owned by linux611
What I find strange here, is the fact that the file does not get removed when removing the kernel.
My system does not have the kernel-modules-hook synced - so the remnants on my system cannot be related to that.
If you are not using kernel-modules-hook
then yes its different.
I think we have at least 2 different issues here.
-
kernel-modules-hook
users with fully populated past kernel modules directories -
Non-users of
kernel-modules-hook
that somehow retain/usr/lib/modules/*/modules.weakdep
In the first case I believe it is solved by the related service (linux-modules-cleanup
).
Which of course makes total sense β¦ If you have some script that fires to copy the modules for the current kernel and the newly installed version - making sure that your current session does not lose modules - when is it going to remove the old kernel modules? Obviously not during the current session - that would defeat the purpose of the script. The script itself cant do the removal now, or on next boot when its no longer running, etc. So there is a service that if ran will clean up the old modules. Or if enabled will check for and remove them on next session.
I dont know about the second case. Its not reflected in my current situation - I am of the first group.
Yup - the latter confuses me - the file left behind clearly belongs to the package in question - why does pacman leave it behind?
The behaviour is recent as I have had all the previous kernels at some point - but only 6.10 leaves the file behind?
One is speculating if the pacman release candidate is causing this?
I wonder about hooks maybe.
uhmm β¦?
pacman -Qo /usr/share/libalpm/hooks/*
grep -l modules /usr/share/libalpm/hooks/*
And then some manual cat
s to see what they do.
I also slapped together this bash line β¦
_hooks2review="./hooks_to_review.txt"; printf '\n Searching and compiling hooks for review.\n\n Please open %s'"$_hooks2review"'\n\n'; printf '\n# These hooks contain reference to "modules."\n# Please Review.\n\n' > $_hooks2review && grep -l modules /usr/share/libalpm/hooks/* | while IFS= read -r line; do printf "\n\n%s""${line}""\n\n" && cat "${line}"; done >> $_hooks2review
here are the hooks to review
# These hooks contain reference to "modules."
# Please Review.
/usr/share/libalpm/hooks/60-depmod.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/modules/*/
Target = !usr/lib/modules/*/?*
[Action]
Description = Updating module dependencies...
When = PostTransaction
Exec = /usr/share/libalpm/scripts/depmod
NeedsTargets
/usr/share/libalpm/hooks/60-mkinitcpio-remove.hook
[Trigger]
Type = Path
Operation = Remove
Target = usr/lib/modules/*/vmlinuz
[Trigger]
Type = Package
Operation = Remove
Target = mkinitcpio
Target = mkinitcpio-git
[Action]
Description = Removing linux initcpios...
When = PreTransaction
Exec = /usr/share/libalpm/scripts/mkinitcpio remove
NeedsTargets
/usr/share/libalpm/hooks/90-mkinitcpio-install.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/initcpio/*
Target = usr/lib/firmware/*
Target = usr/src/*/dkms.conf
Target = usr/lib/systemd/systemd
Target = usr/bin/cryptsetup
Target = usr/bin/lvm
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Target = usr/lib/modules/*/vmlinuz
[Trigger]
Type = Package
Operation = Install
Operation = Upgrade
Target = mkinitcpio
Target = mkinitcpio-git
[Action]
Description = Updating linux initcpios...
When = PostTransaction
Exec = /usr/share/libalpm/scripts/mkinitcpio install
NeedsTargets
/usr/share/libalpm/hooks/99-update-grub.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/modules/*/vmlinuz
Target = boot/vmlinuz*
[Action]
Description = Updating Grub-Bootmenu
When = PostTransaction
Exec = /usr/bin/update-grub
/usr/share/libalpm/hooks/detect-old-perl-modules.hook
[Trigger]
Operation = Install
Operation = Upgrade
Type = Path
Target = usr/lib/perl5/*/
[Action]
Description = Warn about old perl modules
When = PostTransaction
Exec = /usr/share/libalpm/scripts/detect-old-perl-modules.sh
/usr/share/libalpm/hooks/gdk-pixbuf-query-loaders-32.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib32/gdk-pixbuf-2.0/2.10.0/loaders/*.so
[Action]
Description = Probing 32-bit GDK-Pixbuf loader modules...
When = PostTransaction
Exec = /usr/bin/gdk-pixbuf-query-loaders-32 --update-cache
/usr/share/libalpm/hooks/gdk-pixbuf-query-loaders.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/gdk-pixbuf-2.0/2.10.0/loaders/*.so
[Action]
Description = Probing GDK-Pixbuf loader modules...
When = PostTransaction
Exec = /usr/bin/gdk-pixbuf-query-loaders --update-cache
/usr/share/libalpm/hooks/gio-querymodules-32.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib32/gio/modules/*.so
[Action]
Description = Updating 32-bit GIO module cache...
When = PostTransaction
Exec = /usr/bin/gio-querymodules-32 /usr/lib32/gio/modules
/usr/share/libalpm/hooks/gio-querymodules.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/gio/modules/*.so
[Action]
Description = Updating GIO module cache...
When = PostTransaction
Exec = /usr/bin/gio-querymodules /usr/lib/gio/modules
/usr/share/libalpm/hooks/gtk4-querymodules.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/gtk-4.0/4.0.0/*/
[Action]
Description = Updating GTK4 module cache...
When = PostTransaction
Exec = /usr/share/libalpm/scripts/gtk4-querymodules
NeedsTargets
/usr/share/libalpm/hooks/gtk-query-immodules-2.0.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/gtk-2.0/2.10.0/immodules/*.so
[Action]
Description = Probing GTK2 input method modules...
When = PostTransaction
Exec = /usr/bin/gtk-query-immodules-2.0 --update-cache
/usr/share/libalpm/hooks/gtk-query-immodules-3.0-32.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib32/gtk-3.0/3.0.0/immodules/*.so
[Action]
Description = Probing 32-bit GTK3 input method modules...
When = PostTransaction
Exec = /usr/bin/gtk-query-immodules-3.0-32 --update-cache
/usr/share/libalpm/hooks/gtk-query-immodules-3.0.hook
[Trigger]
Type = Path
Operation = Install
Operation = Upgrade
Operation = Remove
Target = usr/lib/gtk-3.0/3.0.0/immodules/*.so
[Action]
Description = Probing GTK3 input method modules...
When = PostTransaction
Exec = /usr/bin/gtk-query-immodules-3.0 --update-cache
I think the offending one is /usr/share/libalpm/hooks/60-depmod.hook
but i cannot fully understand the script
[teo@teo-lenovo-v15 ~]$ cat /usr/share/libalpm/scripts/depmod
#!/bin/bash
while read -r f; do
if [[ -e ${f}modules.order ]]; then
depmod $(basename "$f")
elif [[ -d $f ]]; then
rm -f "${f}"modules.{alias,alias.bin,builtin.alias.bin,builtin.bin} \
"${f}"modules.{dep,dep.bin,devname,softdep,symbols,symbols.bin}
rmdir --ignore-fail-on-non-empty "$f"
fi
done
# vim:set ft=sh sw=2 et:
for example modules.order
should also have been left from the cleaning but it is not.
Maybe this line of /usr/share/libalpm/hooks/60-depmod.hook
:
should be changed to:
"${f}"modules.{dep,dep.bin,devname,softdep,symbols,symbols.bin,weakdep}
Edit (since I canβt post 2 replies in a row):
This kernel.org thread might also be helpful in assessing what, if any, action should be taken. The thread is from the end of July, 2024:
https://lore.kernel.org/lkml/CAK7LNATMauHO_vqr+3LKyLWw9AZ2Y=dxhbgaaVNjdz_qm5Xm=w@mail.gmail.com/T/#u
It is certainly a recent change because i have only 1 old version, from August. So that must be it, but i do not fully understand what is it it and how it works so i would not touch it with a broomstick
But as said, if that is the cleaning filter, another 3 files are not covered with it, and they are gone. So there is maybe something else.
You guys are the best.
Sorry didnβt mean to uncover a can of worms.
So to summarize for me at least.
I have kernel-modules-hook and if I run the cleanup service I should be good to go and not have this issue anymore?
Thanks
Report back (in a day or so) to confirm whether or not it worked for you.
To add to what already have been discovered, my modules folder looks like this:
> tree -L2 -v /usr/lib/modules/
/usr/lib/modules/
βββ 5.10.63-1-MANJARO
β βββ extra
β βββ kernel
βββ 5.13.15-1-MANJARO
β βββ extra
β βββ kernel
βββ 5.15.76-1-MANJARO
β βββ updates
βββ 5.15.84-1-MANJARO
β βββ updates
βββ 5.15.85-1-MANJARO
β βββ kernel
β βββ updates
βββ 5.17.1-3-MANJARO
β βββ build
β βββ extramodules -> ../extramodules-5.17-MANJARO
β βββ kernel
β βββ kernelbase
β βββ modules.alias
β βββ modules.alias.bin
β βββ modules.builtin
β βββ modules.builtin.alias.bin
β βββ modules.builtin.bin
β βββ modules.builtin.modinfo
β βββ modules.dep
β βββ modules.dep.bin
β βββ modules.devname
β βββ modules.order
β βββ modules.softdep
β βββ modules.symbols
β βββ modules.symbols.bin
β βββ pkgbase
β βββ vmlinuz
βββ 5.18.17-1-MANJARO
β βββ build
β βββ extra
β βββ extramodules -> ../extramodules-5.18-MANJARO
β βββ kernel
β βββ kernelbase
β βββ modules.alias
β βββ modules.alias.bin
β βββ modules.builtin
β βββ modules.builtin.alias.bin
β βββ modules.builtin.bin
β βββ modules.builtin.modinfo
β βββ modules.dep
β βββ modules.dep.bin
β βββ modules.devname
β βββ modules.order
β βββ modules.softdep
β βββ modules.symbols
β βββ modules.symbols.bin
β βββ modules.weakdep
β βββ pkgbase
β βββ updates
β βββ vmlinuz
βββ 5.19.17-1-MANJARO
β βββ updates
βββ 6.0.14-1-MANJARO
β βββ updates
βββ 6.6.46-1-MANJARO
β βββ modules.weakdep
βββ 6.6.47-1-MANJARO
β βββ build
β βββ extramodules
β βββ kernel
β βββ kernelbase
β βββ modules.alias
β βββ modules.alias.bin
β βββ modules.builtin
β βββ modules.builtin.alias.bin
β βββ modules.builtin.bin
β βββ modules.builtin.modinfo
β βββ modules.dep
β βββ modules.dep.bin
β βββ modules.devname
β βββ modules.order
β βββ modules.softdep
β βββ modules.symbols
β βββ modules.symbols.bin
β βββ modules.weakdep
β βββ pkgbase
β βββ updates
β βββ vmlinuz
βββ 6.10.5-1-MANJARO
β βββ modules.weakdep
βββ 6.10.6-10-MANJARO
βββ build
βββ extramodules
βββ kernel
βββ kernelbase
βββ modules.alias
βββ modules.alias.bin
βββ modules.builtin
βββ modules.builtin.alias.bin
βββ modules.builtin.bin
βββ modules.builtin.modinfo
βββ modules.dep
βββ modules.dep.bin
βββ modules.devname
βββ modules.order
βββ modules.softdep
βββ modules.symbols
βββ modules.symbols.bin
βββ modules.weakdep
βββ pkgbase
βββ updates
βββ vmlinuz
38 directories, 71 files
I have a bunch of really old stuff (and only 6.6.47-1
and 6.10.6-10
are installed right now).
The older kernels seems to be about something already fixed.
The modules.weakdep
thing seems to be quite recent. It only affects my previous kernels.
> grep -P '(installed|removed|upgraded) linux66 ' /var/log/pacman.log
...omitted for brevity...
[2024-07-29T07:33:43+0300] [ALPM] upgraded linux66 (6.6.40-1 -> 6.6.41-1)
[2024-08-08T19:43:18+0300] [ALPM] upgraded linux66 (6.6.41-1 -> 6.6.44-1)
[2024-08-22T19:48:50+0300] [ALPM] upgraded linux66 (6.6.44-1 -> 6.6.46-1)
[2024-09-03T08:48:11+0300] [ALPM] upgraded linux66 (6.6.46-1 -> 6.6.47-1)
> grep -P '(installed|removed|upgraded) linux610 ' /var/log/pacman.log
[2024-08-08T19:45:59+0300] [ALPM] installed linux610 (6.10.3-1)
[2024-08-22T19:48:47+0300] [ALPM] upgraded linux610 (6.10.3-1 -> 6.10.5-1)
[2024-09-03T08:48:08+0300] [ALPM] upgraded linux610 (6.10.5-1 -> 6.10.6-10)
I do NOT have installed kernel-modules-hook
nor kernel-alive
.
No need to apologise for reporting an issue that may affect many users
I had a similar problem. Installed kernel-modules-hook
and used linux-modules-cleanup
to remove old modules
$ systemctl status linux-modules-cleanup
β linux-modules-cleanup.service - Clean up modules from old kernels
Loaded: loaded (/usr/lib/systemd/system/linux-modules-cleanup.service; disabled; preset: disabled)
Active: inactive (dead)
Sep 29 21:57:01 gnomic bash[6686]: + for i in /usr/lib/modules/[0-9]*
Sep 29 21:57:01 gnomic bash[6686]: + [[ 6.6.52-1-MANJARO = \6\.\1\.\1\1\1\-\1\-\M\A\N\J\A\R\O ]]
Sep 29 21:57:01 gnomic bash[6686]: + pacman -Qo /usr/lib/modules/6.6.52-1-MANJARO
Sep 29 21:57:01 gnomic bash[6705]: /usr/lib/modules/6.6.52-1-MANJARO/ is owned by linux66 6.6.52-1
Sep 29 21:57:01 gnomic bash[6705]: /usr/lib/modules/6.6.52-1-MANJARO/ is owned by linux66-headers 6.6.52-1
Sep 29 21:57:01 gnomic bash[6705]: /usr/lib/modules/6.6.52-1-MANJARO/ is owned by linux66-nvidia-470xx 470.256.02-23
Sep 29 21:57:01 gnomic bash[6686]: + continue
Sep 29 21:57:01 gnomic systemd[1]: linux-modules-cleanup.service: Deactivated successfully.
Sep 29 21:57:01 gnomic systemd[1]: Finished Clean up modules from old kernels.
Sep 29 21:57:01 gnomic systemd[1]: linux-modules-cleanup.service: Consumed 763ms CPU time, 102.6M memory peak
But β¦ like β¦
The issue is that if using kernel-modules-hook
β¦ you need to also use the included service or else these fully populated modules folders continue to exist.
The other issue is users not using kernel-modules-hook
that for some other reason retain modules.weakdep
files (the old modules folders are not fully populated).
I suppose you can use the service from kernel-modules-hook
to automatically remove the old modules folders β¦ but thats β¦ fixing issue 2 by introducing and then solving issue 1.
You can start
the service to run it once immediately.
For βnever think about this againβ solution then you should enable it.
systemctl enable linux-modules-cleanup --now
(it will run each login and check for old modules folders, and delete if they exist will move them to .old
in that directory, and .old
will be cleaned by systemd-tmpfiles-clean.service
as defined by the config file /usr/lib/tmpfiles.d/linux-modules-cleanup.conf
)
Mine arenβt deleted. They were moved to a hidden folder /usr/lib/modules/.old
This really is not ideal and Iβm sure itβs a bug. Anyway, I think Iβm just going to manually delete them.
Sorry, I should have been more explicit.
This is controlled by /usr/lib/tmpfiles.d/linux-modules-cleanup.conf
;
R! /usr/lib/modules/.old/* - - - 4w
Which is defined like here:
https://www.man7.org/linux/man-pages/man5/tmpfiles.d.5.html
And so will be taken care of according to that configuration and systemd-tmpfiles
and/or the systemd-tmpfiles-clean.service
and/or systemd-tmpfiles-clean.timer
.
You can also run**
sudo systemd-tmpfiles --remove
** In fact this wont remove it with the default conf. Due to the !
in R!
the removal can only happen during boot.
Which will do it right away - for things in .old
that are older than 4 weeks, with the default conf. You can edit it to 12h for 12 hours and so on.
I did not introduce an issue. Issue occurred after latest Testing branch update
Data posted for cleanup service you suggested to confirm it works as expected
I am on Testing branch with pacman
6.1.0-7
I had old versions of modules.weakdep
for kernels 6.1 and 6.6 but not for 6.10
Partnerβs system on stable branch had similar issue for both installed kernels 6.1 and 6.6
$ ls -al /usr/lib/modules
total 264
drwxr-xr-x 6 root root 4096 Sep 6 11:37 .
drwxr-xr-x 244 root root 241664 Sep 6 11:37 ..
drwxr-xr-x 2 root root 4096 Sep 6 11:37 6.1.105-1-MANJARO
drwxr-xr-x 4 root root 4096 Sep 6 11:37 6.1.106-1-MANJARO
drwxr-xr-x 2 root root 4096 Sep 6 11:37 6.6.46-1-MANJARO
drwxr-xr-x 4 root root 4096 Sep 6 11:37 6.6.47-1-MANJARO
Old folders deleted manually