One year later and it seems Manjaro still doesn’t have a proper solution. Just encountered a crash and want to report it with a proper stack trace, but no cigar
I understand that, from a developer’s point of view, it is desirable to only have bug reports against the latest version. But from an end user perspective, who needs a stable working environment, it is not really feasible to switch to unstable just to be able to try to recreate the crash and report it. I cannot reproduce the crash at will, so it would have been nice to be able to analyze the problem and report it to the developers.
I’m just wondering if it’s a technical problem preventing a solution, or if it’s limited resources (time, money, …) or low priority or simply won’t be done because Manjaro firmly believes that debug symbols for all stable packages are simply not needed?
if you have an issue with a package on stable branch and a newer version is in testing or unstable - your should switch branch to determine if the malfunction is still present.
Only then debug symbols make sense.
if there is no newer version then use the debuginfo from Arch.
If the application you want to debug has a newer version in unstable - which is equivalent of Arch stable - then it is necessary.
The thing is submitting bug reports for outdated/unsupported software may be a waste of devs time, the bug may already be fixed or isn’t relevant anymore, or is known but not yet fixed.
Plus they’ll ask you to update anyway, so you might as well update first and then submit the bug report if needed.
I understand that the bug might be already known or even fixed, but this can be the case for bug reports on the most recent version as well. You don’t expect users to build and install the latest version in development from git every few hours just in case a bug has already been fixed, or do you?
I want to be able to create a decent stack trace with symbols and line numbers from stable software. I can then check the already submitted bug reports to see if it’s a known bug or not, before submitting. An enthusiastic user with time at hand and development knowledge might even be inclined to have a look at the source and be able to figure out what went wrong, but this cannot be done without a proper stack trace.
I do not agree with that statement. IMHO, debug symbols make sense in every environment. I cannot get a proper stack trace without them. I need this stack trace to be able to check existing bug reports to see if it’s a known bug. I also like to take a look at the source code and try to figure out what went wrong.
The only case where debug symbols make not much sense is if the source of the version I’m running is not available.
I tried using debuginfod from Arch, but that resulted in a useless stack trace
#0 0x0000730b86e7d473 in Plasma::Theme::~Theme() () from /usr/lib/libPlasma.so.6
#1 0x0000730b77e08c75 in ?? () from /usr/lib/qt6/plugins/kf6/kirigami/platform/KirigamiPlasmaStyle.so
#2 0x0000730b8458c60b in QObjectPrivate::deleteChildren() () from /usr/lib/libQt6Core.so.6
#3 0x0000730b84592678 in QObject::~QObject() () from /usr/lib/libQt6Core.so.6
#4 0x0000730b77e31da2 in ?? () from /usr/lib/qt6/qml/org/kde/ksvg/libcorebindingsplugin.so
#5 0x0000730b8458c60b in QObjectPrivate::deleteChildren() () from /usr/lib/libQt6Core.so.6
#6 0x0000730b84592678 in QObject::~QObject() () from /usr/lib/libQt6Core.so.6
#7 0x0000730b85fd6972 in ?? () from /usr/lib/libQt6Quick.so.6
#8 0x0000730b8458d68a in QObject::event(QEvent*) () from /usr/lib/libQt6Core.so.6
#9 0x0000730b866fc8cc in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib/libQt6Widgets.so.6
#10 0x0000730b84545aa8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib/libQt6Core.so.6
#11 0x0000730b84545e6b in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /usr/lib/libQt6Core.so.6
#12 0x0000730b847aa00c in ?? () from /usr/lib/libQt6Core.so.6
#13 0x0000730b832bc299 in ?? () from /usr/lib/libglib-2.0.so.0
#14 0x0000730b8331eec7 in ?? () from /usr/lib/libglib-2.0.so.0
#15 0x0000730b832bb795 in g_main_context_iteration () from /usr/lib/libglib-2.0.so.0
#16 0x0000730b847a82bd in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt6Core.so.6
#17 0x0000730b8454ff66 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt6Core.so.6
#18 0x0000730b8454a11d in QCoreApplication::exec() () from /usr/lib/libQt6Core.so.6
#19 0x00005e1f1c1e5d86 in ?? ()
#20 0x0000730b84a34e08 in __libc_start_call_main (main=main@entry=0x5e1f1c1e2e10, argc=argc@entry=2, argv=argv@entry=0x7ffd42cafb08) at ../sysdeps/nptl/libc_start_call_main.h:58
#21 0x0000730b84a34ecc in __libc_start_main_impl (main=0x5e1f1c1e2e10, argc=2, argv=0x7ffd42cafb08, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd42cafaf8) at ../csu/libc-start.c:360
#22 0x00005e1f1c1e6275 in ?? ()
post edited to show correct attribution of quotations - nikgnomic
The vast majority of packages in Manjaro Linux repos is build by Arch Linux. For debug symbols you will have get the symbols from Arch LInux.
If you want to create stacktrace for a given application on Manjaro - you will need to use unstable branch as debug symbols will only match package on unstable branch.
I don’t think it is reasonable to expect users to spend hours or even days to download, compile and install a huge amount of libraries and applications just to be able to create a proper stack trace for a random crash, I cannot reproduce at will. IMHO, it should be made as easy as possible to create a proper bug report.
I need a stable environment to work with. I, and probable most other users on stable, don’t have the time and/or knowledge to deal with all the potential instabilities on the unstable branch. I know that some people on unstable say they have been running unstable for years without problems, but there are also a lot of comments from users on unstable reporting issues.
If Manjaro requires everyone to use the unstable branch then there is not really a point having testing and stable branches.
I encountered a random bug on stable. I want to help analyze the problem to improve Manjaro stable and the software it provides us with, but cannot do it.
I understand that Manjaro is free. I am grateful for this and beggars can’t be choosers, but quite a few (most?) of the other distributions do provide debug information and thus enable their users to help improve the distribution by providing proper bug reports with minimal effort. I wondered why Manjaro chooses not to provide debug info for stable.
We all use free and open source software. We all profit from it and some of us even improve it, either by submitting bug reports, patches for new functionality or bug fixes. Even the original developers profit from this by having better and more stable software.
While being on Manjaro stable I cannot give back to the community by submitting proper bug reports or even bug fixes, because I cannot easily get a proper stack trace. I know that I could download the src and build everything myself. I could also ditch Manjaro completely and build and install my own Linux from scratch, but I am not going to do this because it would take way to much time and effort.
The result of the current situation is that I am not able to analyze the random crash I got on stable, am not able to submit a bug report or even a patch with a fix and hence not give back to the community. I think there is potential in Manjaro stable users being able to give back, that is not used. I simply wondered why this is the case:
Stable branch has not been updated since 2024-10-10 so unknown package may need an update - Branch compare for Manjaro
Manjaro does not have a debuginfod server for stable branch packages maintained by Manjaro
For packages inherited from Arch; issues (or bug reports with data) must be reported upstream to source developers - Arch maintainers do not accept reports from Manjaro
Yes, I figured as much. My question was, why, and what, if anything can be done about it.
I wasn’t going to report the issue to Manjaro or Arch, as the crash of that application was most likely not caused by a packaging issue, but a bug in the code. But without a stack trace I cannot even begin to investigate and don’t need to bother reporting an a random crash to an upstream developer.
No need to ridicule or belittle me. A stack trace is certainly not the only means to analyze a random problem, but it is easy to use and usually the first place to start an investigation. That’s why almost all developers (rightly so) require a proper stack trace for a bug report that cannot be reliably reproduced. I don’t know any other QUICK way that allows me to find the cause for the crash, but I am open for suggestions and willing to learn. So, please tell me how I can figure out without spending countless hours or even days what went wrong with this application. All I have is the core file,
The name is just a name, used to signal the difference between the branches.
Everyone does - I get that - what makes you think that unstable/testing branch is more unstable than stable?
But as there often is a package versioning gap between stable and testing, if you need a stacktrace for a given application you will need to use the correct version to match debug symbols. And yes - Manjaro does not provide debug symbols for generic applications, with one exception - Pamac has versions compiled with symbols.
This is not rocket science - if there is a versioning gap between stable and testing/unstable - then you may expect that the very bug you are trying to trace has been fixed.
When packages are imported from Arch repos - they already been through a testing phase and is considered stable - and the debug symbols match those packages.
Only a small subset of those packages are (re)built by a Manjaro maintainer.
Those built by a Manjaro team member is either Manjaro utilities or they are created to cater for a specific issue(s) the team want to fend of before reaching stable branch.
One example is the default of X11 as opposed to Wayland when it comes to Plasma Desktop.
Other examples are the kernels and the matching Nvidia and VirtualBox drivers.
If you want to know which applications Manjaro Linux provides overlay for - they are listed in the repo see your primary mirror e.g. Index of /manjaro/pool/overlay
Hmm, for one the name Usually new stuff is first brought to unstable, then testing, and then stable. New stuff usually means new features and bug fixes, but unfortunately also new bugs. Software is usually only promoted to testing and later to stable if most of the new kinks have been fixed. This is, as I understand it, one of the reasons that bigger DE updates like Plasma 5 → Plasma 6, or even smaller ones to 6.1 and 6.2 appear and stay in unstable and testing for weeks and even months before being promoted to stable. That makes me think that stable is more stable than unstable. Some users are more eager to run the latest and greatest versions and willing to accept potential instabilities. Some might have separate systems simply for testing. I am grateful to those users, as they help getting the software more stable before it is promoted to stable. I used to be one of them but I don’t have the time anymore and have to focus more on my productivity.
Hmm, not sure I understand this. I’m using pamac to install and update all my packages. Can I somehow select that I want to install all packages with symbols? If so, how?
My experience is a different one. I may expect a bug has been fixed, but given the long list of open bugs for most open source repositories makes me doubt it will have been. But hoping or not is besides the point. If I had a stack trace, I could check the open bug reports and also if the code has been changed where the bug occurred. This would have been my part of contributing to the community making the software more stable and better for everyone, with limited time and effort required on my side.
That is interesting and helpful information, but it still does not answer my question.
… and is there anything that the community can help with to change this?
No - that is not what I meant - the pamac package is provided in two versions a normal version and a -debug version
$ pamac search pamac --no-aur
libpamac-snap-plugin 11.7.0-1 extra
Snap plugin for Pamac
libpamac-flatpak-plugin 11.7.0-1 extra
Flatpak plugin for Pamac
libpamac-debug 11.7.0-1 [Installed] extra
Detached debugging symbols for libpamac
libpamac 11.7.0-1 [Installed] extra
Library for Pamac package manager based on libalpm
pamac-gtk 11.7.2-1 extra
A Package Manager based on libalpm with AUR and Appstream support (GTK4)
pamac-gnome-integration 11.7.2-1 extra
Pamac GNOME integration (GTK3)
pamac-debug 11.7.2-1 extra
Detached debugging symbols for pamac
pamac-tray-icon-plasma 0.1.3-3 [Installed] extra
Pamac tray icon for Plasma users
pamac-gtk3-debug 10.7.0-1 [Installed] extra
Detached debugging symbols for pamac-gtk3
pamac-gtk3 10.7.0-1 [Installed] extra
A Package Manager based on libalpm with AUR and Appstream support (GTK3)
pamac-cli-debug 11.7.0-2 [Installed] extra
Detached debugging symbols for pamac-cli
pamac-cli 11.7.0-2 [Installed] extra
A CLI Package Manager based on libalpm with AUR support
I understood your question as -
Does Manjaro provide debug symbols for stable packages
The answer is no - except pamac which is provided in two versions - symbols is detached in a separate package.
If you want debug symbols for other packages - understood as imported from Arch (no overlay) - you will need verify if the version in Manjaro repo of choice is identical to the version in Arch stable repo.
If the version in stable is identical to Arch then debug symbols can be obtained from Arch
You can get this info using manjaro-check-repos package (run mbn update to ensure your db’s are up-to-date).
Example
$ mbn info firefox -q | grep -e 'Branch' -e 'Version' -e 'Packager'
Branch : archlinux
Version : 132.0.1-1
Packager : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Branch : unstable
Version : 132.0.1-1
Packager : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Branch : testing
Version : 132.0-1
Packager : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Branch : stable
Version : 131.0.3-1
Packager : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
$ coredumpctl
TIME PID UID GID SIG COREFILE EXE SIZE
Tue 2024-11-05 15:04:51 CET 467663 1000 1000 SIGSEGV present /usr/bin/plasmashell 38.6M
Wed 2024-11-06 13:51:11 CET 865155 1000 1000 SIGABRT present /usr/bin/strawberry 265.9M
$ pamac search -f /usr/bin/plasmashell
/usr/bin/plasmashell is owned by plasma-workspace
So the packages in question are
plasma-workspace 6.1.5-1.0 [Installed]
strawberry 1.1.2-2 [Installed]
Let’s focus on the plasma issue first. I connected an Amazon tablet to a USB port, which I have done countless times before without any issues. Then plasmashell simply crashed while I was using the computer. journalctl shows the following entries shortly before the crash
Nov 05 15:04:41 server1 plasmashell[467663]: kf5idletime_wayland: This plugin does not support polling idle time
Nov 05 15:04:46 server1 kernel: usb 3-3: new high-speed USB device number 71 using xhci_hcd
Nov 05 15:04:46 server1 kernel: usb 3-3: New USB device found, idVendor=1949, idProduct=06b1, bcdDevice= 4.04
Nov 05 15:04:46 server1 kernel: usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Nov 05 15:04:46 server1 kernel: usb 3-3: Product: KFSNWI
Nov 05 15:04:46 server1 kernel: usb 3-3: Manufacturer: Amazon
Nov 05 15:04:46 server1 kernel: usb 3-3: SerialNumber: <Deleted>
Nov 05 15:04:46 server1 mtp-probe[828047]: checking bus 3, device 71: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-3"
Nov 05 15:04:46 server1 mtp-probe[828047]: bus: 3, device: 71 was an MTP device
Nov 05 15:04:46 server1 kiod6[10600]: Device 0 (VID=1949 and PID=06b1) is UNKNOWN in libmtp v1.1.21.
Nov 05 15:04:46 server1 kiod6[10600]: Please report this VID/PID and the device model to the libmtp development team
Nov 05 15:04:46 server1 kiod6[10600]: Device 1 (VID=1949 and PID=06b1) is UNKNOWN in libmtp v1.1.21.
Nov 05 15:04:46 server1 kiod6[10600]: Please report this VID/PID and the device model to the libmtp development team
Nov 05 15:04:46 server1 mtp-probe[828073]: checking bus 3, device 71: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-3"
Nov 05 15:04:46 server1 mtp-probe[828073]: bus: 3, device: 71 was an MTP device
Nov 05 15:04:47 server1 kernel: plasmashell[467663]: segfault at b68001700 ip 0000730b86e7d473 sp 00007ffd42caf020 error 6 in libPlasma.so.6.1.5[40473,730b86e52000+37000] likely on CPU 1 (core 1, socket 0)
Nov 05 15:04:47 server1 kernel: Code: 55 41 54 53 48 89 fb 48 83 ec 08 48 8b 05 35 ba 01 00 48 83 c0 10 48 89 07 48 8b 47 10 48 3b 05 bb dc 01 00 0f 84 ed 00 00 00 <f0> 83 68 10 01 74 16 48 83 c4 08 48 89 df 5b 41 5c 41 5d 5>
Nov 05 15:04:47 server1 systemd-coredump[828075]: Process 467663 (plasmashell) of user 1000 terminated abnormally with signal 11/SEGV, processing...
Nov 05 15:04:47 server1 systemd[1]: Started Process Core Dump (PID 828075/UID 0).
Nov 05 15:04:51 server1 systemd-coredump[828076]: [🡕] Process 467663 (plasmashell) of user 1000 dumped core.
Stack trace of thread 467663:
#0 0x0000730b86e7d473 _ZN6Plasma5ThemeD2Ev (libPlasma.so.6 + 0x40473)
Usually new stuff is first brought to unstable, then testing, and then stable. New stuff usually means new features and bug fixes, but unfortunately also new bugs. Software is usually only promoted to testing and later to stable if most of the new kinks have been fixed. This is, as I understand it, one of the reasons that bigger DE updates like Plasma 5 → Plasma 6, or even smaller ones to 6.1 and 6.2 appear and stay in unstable and testing for weeks and even months before being promoted to stable. That makes me think that stable is more stable than unstable. Some users are more eager to run the latest and greatest versions and willing to accept potential instabilities. Some might have separate systems simply for testing. I am grateful to those users, as they help getting the software more stable before it is promoted to stable. I used to be one of them but I don’t have the time anymore and have to focus more on my productivity.