Update failure with "failed to commit transaction (invalid or corrupted package)"

I am using Manjaro Stable Branch.

Update failed today with this screen output

:: Import PGP key 3D4C5008BB5C8D29, "Peter Jung <ptr1337@archlinux.org>"? [Y/n] y
(653/653) checking package integrity                   [############################] 100%
error: python-charset-normalizer: signature from "Daniel M. Capella <polyzen@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-charset-normalizer-3.4.0-1-any.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: python-jaraco.collections: signature from "Daniel M. Capella <polyzen@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-jaraco.collections-5.0.1-1-any.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: python-setuptools: signature from "Daniel M. Capella <polyzen@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-setuptools-1:75.2.0-2-any.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: failed to commit transaction (invalid or corrupted package)
Errors occurred, no packages were upgraded.

Here are some surrounding facts

  • My Manjaro machines run in VM. Four of the five such VM machines updated successfully. Those four also gave Import PGP key 3D4C5008BB5C8D29, "Peter Jung <ptr1337@archlinux.org>"? [Y/n], and upon my pressing y updated fine.
  • This 5th and failing one (affected machine) is actually the one I try to keep in a pristine state by not using it. (Each of the four other machines might have some extra packages that others donā€™t. But the affected one has nothing extra.)
  • I have successfully updated the affected machine on these dates: 24-08-31, 24-09-07, 24-10-27. The machineā€™s condition on those dates immediately after update was preserved in a snapshot.
  • I naturally tried to update from 24-10-27 first and got the above quoted screen output.
  • I then restored the machine to its 24-09-07 state, tried to update, and got the same output.
  • Finally I restored the machine to its 24-08-31 state and got the same result.
  • Notice that the two earlier dates 24-08-31 and 24-09-07 have a record of having successfully served as points from which to update. (I.e. from 24-08-31 to 24-09-07, and from 24-09-07 to 24-10-27). But now they can no longer serve as such points.
  • The affected machine exist in 2 copies. I tried both (one only from its 24-10-27 state), and they both fail in the same way.

What can I do other than

  • trying to update from an earlier snapshot
  • using one of the four machines that do update as template to create a new fifth machine?

Thanks.

You could try a forced refresh of the package db.

pamac upgrade -a --force-refresh --dry-run

ā€¦orā€¦

pamac upgrade -a --force-refresh

if the first command fails to produce any joy.

With all those restores you will probably want to sync as soon as possible using pacman.


Nothing else to add at this time, but others might.

Regards.

2 Likes

The second command, i.e.,

pamac upgrade -a --force-refresh

worked. Thank you very much!

Just for anybody having a similar problem, the first command produced a moderate (and therefore unpromising-looking) screen output as follows:

[luna@jar ~]$ pamac upgrade -a --force-refresh --dry-run
Preparing...
** Message: 16:45:53.438: aur_plugin.vala:332: downloading AUR data
Checking pulseaudio-ctl dependencies...
Resolving dependencies...
Checking inter-conflicts...

To install (12):
  m4              1.4.19-3  (Required By: base-devel)  core  251.9 kB
  autoconf        2.72-1    (Required By: base-devel)  core  666.1 kB
  automake        1.17-1    (Required By: base-devel)  core  641.7 kB
  bison           3.8.2-8   (Required By: base-devel)  core  784.7 kB
  debugedit       5.0-6     (Required By: base-devel)  core  45.1 kB
  flex            2.6.4-5   (Required By: base-devel)  core  314.9 kB
  gc              8.2.8-2   (Required By: base-devel)  core  240.2 kB
  guile           3.0.10-1  (Required By: base-devel)  core  8.7 MB
  make            4.4.1-2   (Required By: base-devel)  core  536.4 kB
  patch           2.7.6-10  (Required By: base-devel)  core  95.2 kB
  pkgconf         2.1.1-1   (Required By: base-devel)  core  61.9 kB
  base-devel      1-3                                  core  22.3 kB
To build (1):
  pulseaudio-ctl  1.70-2    (1.70-1)                   AUR

Total download size: 12.4 MB
Total installed size: 68.5 MB

Transaction successfully finished.
[luna@jar ~]$

The second commandā€™s output is too massive to reproduce in its totality, but it ended with:

Total download size: 1.8 GB
Total installed size: 226.2 MB
Total removed size: 3.5 MB

Edit build files : [e] 
Apply transaction ? [e/y/N] 

To which I answered: y.

Then it went through a process that looked much like my regular updating process plus more stuff and ended with Transaction successfully finished.

When I ran sudo pacman -Syu, I got an output ending with there is nothing to do.

A couple questions:

  1. Am I right to think that what pamac upgrade -a --force-refresh did was to download current versions of all the packages I have?
  2. What could have gone wrong with those ā€œcorruptedā€ python-related packagesā€“especially as they were in a machine locked away on an unconnected storage device for safekeeping?

The first command was the same as the second.
With the only difference being that only the second actually did anything.
First was a dry run - do nothing but tell what is going to be done.

The second one then actually built an AUR package: pulseaudio-ctl
(thus the long output from compilation that you mentioned)
The package might have been in the repos - I donā€™t know.
Now it isnā€™t - itā€™s from AUR.

System is not quite in the pristine state with nothing added as you said
as you do have at least this AUR package installed.

Was your initial update attempt made with pacman -Syu?

My guess is that this was related to the removal and then restoration of SyncFirst in the default /etc/pacman.conf.

I believe that pamac will sync keyrings prior to all other packages, but pacman will not if the SyncFirst line is not present.

You may have been able to just run sudo pacman -Sy archlinux-keyring first. Daniel Capellaā€™s key was updated in the archlinux-keyring 20241015-1 package, but the Stable branch did not receive it until the 2024-11-30 update.

Thatā€™s interesting. I never consciously went and installed any AUR. On the contrary I remember making a conscious decision to stay off AUR. Of course something else I did might have had the consequence of installing some AUR for me.

Yes. Thanks for the hypothesis (though it rather goes above my head).

You can check for the optionā€™s presence with

grep SyncFirst /etc/pacman.conf

If the option is present, you may get something like

SyncFirst    = manjaro-system archlinux-keyring manjaro-keyring

If the option is not present, you will get nothing back. If so, you can either:

  • Add it yourself manually, or
  • Always check what packages are being updated, and if you see one of the keyrings, update them first separately. This is the standard Arch method.

That was the case for me. When you say update the keyrings first separately,

  1. I understand thatā€™s sudo pacman -Sy archlinux-keyring.
  2. Can I just always run that before sudo pacman -Syu (trusting that if there are no keyrings to update, then the command would have no effect)? Thanks

The deprecated option SyncFirst has not been available since April 2024. Including it in /etc/pacman.conf will just result in an error message:

run0 pacman -Syu
warning: config file /etc/pacman.conf, line 19: directive ā€˜SyncFirstā€™ in section ā€˜optionsā€™ not recognized.
:: Synchronizing package databasesā€¦
core is up to date
extra is up to date
:: Starting full system upgradeā€¦
there is nothing to do

A previous forum post regarding this:

And the Manjaro commit made in April:

1 Like

I donā€™t know whether what follows means anything. But as I said in the original post, the four other VMs could update fine, and that was on Dec 17.

In one of those four machines, I ran the command for updating the keyring and got:

[luna@jar-m ~]$ sudo pacman -Sy archlinux-keyring
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
warning: archlinux-keyring-20241203-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) archlinux-keyring-20241203-1

Total Installed Size:  1.67 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                         [############################] 100%
(1/1) checking package integrity                       [############################] 100%
(1/1) loading package files                            [############################] 100%
(1/1) checking for file conflicts                      [############################] 100%
(1/1) checking available disk space                    [############################] 100%
:: Running pre-transaction hooks...
(1/1) Creating Timeshift snapshot before upgrade...
==> skipping timeshift-autosnap due skipRsyncAutosnap in /etc/timeshift-autosnap.conf set to TRUE.
:: Processing package changes...
(1/1) reinstalling archlinux-keyring                   [############################] 100%
==> Appending keys from archlinux.gpg...
==> Updating trust database...
gpg: next trustdb check due at 2025-01-01
:: Running post-transaction hooks...
(1/2) Reloading system manager configuration...
(2/2) Arming ConditionNeedsUpdate...
[luna@jar-m ~]$ 

Looks as thought thereā€™d been an update between 17th and today (20th where I am).

So itā€™s possible that the four machines that updated on 17th, if I waited to update them till 20th, might also have failed.

It would be a waste of effort to do that every time, I think; but certainly when you know a keyring package is available (hint: check the relevant Update announcements before you update) the following would produce the expected result;

the sync/update would still be performed whether or not the included package was available:

sudo pacman -Syu archlinux-keyring

The community repo hasnā€™t existed for over a year (there was a .pacnew file provided at that time that could have been merged.

You should probably form a strategy for handing your .pacnew files. See the following for a brief introduction:

In particular, look for the heading Pacnew and Pacsave files.

Regards.

2 Likes

Thanks for actually testing. My understanding was that the SyncFirst deprecation was reverted in version 6.1.0-8, but I did not realize that it had been reintroduced in 7.0.0.r3.g7736133.

Why not just always make an informed decision instead?

I canā€™t quite agree with this. The -u part will attempt to update all relevant packages, including those signed by keys that would not be updated until the new keyring is fully installed, so the integrity check will fail. There is a reason that the Arch wiki recommends

# pacman -Sy --needed archlinux-keyring && pacman -Su

for this particular case.

3 Likes

According to the manpage,

ā€“needed
Do not reinstall the targets that are already up-to-date.

-u, --upgrades
Restrict or filter output to packages that are out-of-date on the local system. Only package versions are used to find outdated packages; replacements are not checked here. This option works best if the sync database is refreshed using -Sy.

Questions

  1. It would appear that the two options do the same thing?
  2. If I ran sudo pacman -Sy archlinux-keyring each time before the regular sudo pacman -Syu (as I suggested earlier and people frowned upon), the first command (without either -u or --needed) would reinstall all keyrings, even including those which were up-to-date?
  3. Then instead I could always run sudo pacman -Syu archlinux-keyring or sudo pacman -Sy --needed archlinux-keyring before sudo pacman -Syu (and not cause an unnecessary installation of already up-to-date keyrings)?

ADDED LATER

It seems -u and --needed are not equivalent. See different outputs below.

[luna@jar-m ~]$ sudo pacman -Sy --needed archlinux-keyring 
[sudo] password for luna: 
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
warning: archlinux-keyring-20241203-1 is up to date -- skipping
 there is nothing to do
[luna@jar-m ~]$ sudo pacman -Syu archlinux-keyring
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
warning: archlinux-keyring-20241203-1 is up to date -- reinstalling
:: Starting full system upgrade...
warning: gwenview: ignoring package upgrade (24.05.2-1 => 24.08.3-1)
resolving dependencies...
looking for conflicting packages...

Packages (1) archlinux-keyring-20241203-1

Total Installed Size:  1.67 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] n
[luna@jar-m ~]$ 

I also see that sudo pacman -Syu archlinux-keyring offers to install the same Packages (1) archlinux-keyring-20241203-1 again, no matter how often the command is repeated (and allowed to go on and install the thing).


Thank you for that. I believe that you are telling me that my system does not know that ā€œcommunityā€ is obsolete and that I must do some manual maintenance to fix that.

I did look at the linked material, but it went rather above my head. If there is some other material that might connect community, pacnew, and pacsave and explain the whole thing I would greatly appreciate being linked to it.

The community repo is only one example where a .pacnew file has been created but left unattended for a very long time. That the community repo hasnā€™t been attended to is a strong indicator that other (more important .pacnew instances) have also been neglected.

Content of a .pacnew file may need to be merged (or even replace) the content of its associated file. However each should be viewed first to determine how each instance should be handled.

There is often information in a respective Update announcement when a .pacnew needs attending to, as was the case with the community repo change over a year ago.

Here is the explanation of Pacnew and Pacsave files in the Arch Wiki:

The link already given (from the Manjaro Wiki) is usually sufficient as a starting point, though I recall seeing many discussions on the general topic within the forum:

After researching the topic, if you still feel itā€™s largely above your head, you could create a new thread and ask for input. This may attract some more detailed information from other Members, and cultivate a better understanding.

Perhaps there are a few guides others are aware of. I know the topic has received much attention from many hoping to avoid dealing with pacnews at all. Iā€™d guess most have been disappointed on that note.

Regards.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.