[HowTo] After a Python interpreter upgrade, reinstall non-working Python packages

The instructions that were here have been re-written and are now here.

5 Likes
This guide seems much longer and convoluted than it needs to be

Please consider removing fluff and extra commands you don’t need (for example, the pamac build command will rebuild the package without needing to remove it or clear caches first).

All you need:

  1. Update packages normally. Reboot if required.

  2. Then rebuild remaining packages:

Using pamac
Rebuild only python packages using pamac
pamac build $(pacman -Qoq /usr/lib/python3.10)
Rebuild all AUR packages that need a rebuild using pamac
pamac install rebuild-detector
pamac build $(checkrebuild|grep -oP '^foreign\s+\K(?!.*-bin$)([\w\.@\+\-]*)$')
Using pikaur
Fix pikaur (if broken)
git clone https://github.com/actionless/pikaur.git && pushd pikaur
python3 ./pikaur.py -Sa --rebuild $(pacman -Qq pikaur)
Install pikaur if not installed
git clone https://github.com/actionless/pikaur.git && pushd pikaur
python3 ./pikaur.py -Sa --rebuild pikaur
Rebuild only python packages using pikaur
pikaur -Sa --rebuild $(pacman -Qoq /usr/lib/python3.10)
Rebuild all AUR packages that need a rebuild using pikaur
sudo pacman -S --needed rebuild-detector
pikaur -Sa --rebuild $(checkrebuild|grep -oP '^foreign\s+\K(?!.*-bin$)([\w\.@\+\-]*)$')

I dont know why the first thought if a normal install fails of pikaur that you should rely on pip or python at all … in this guide on how to deal with new/broken python.

There is no mention of

git clone https://aur.archlinux.org/paru.git
cd paru
makepkg -sric

Or similiar, though that should be one of the first attempts.

The guide is mainly about Python installations and pip. A lot of people asking in the forums when we upgraded to 3.10 seemed to have no idea about them, and that motivated me to write this guide.

I don’t really know enough about AUR, compiling C code, shell scripting etc, and asked for feedback earlier. (Not in the right place I guess, as I didn’t get much.)

True, but I’m not sure that it will always recompile for the new Python. I thought that if the same version is in its cache, it might just grab the package it compiled previously for Python 3.10 and drop it into Python 3.11.

Also I’m not sure if it will remove the old package from 3.10 site-packages. If it doesn’t, then when you run pip, pip will re-install those same packages (under $HOME) and then you’ll have 2 copies installed.

So, for the cost of a few seconds running 2 extra commands, I made sure to avoid those negative possibilities.

I don’t use pamac, run only stable branch, and didn’t have time to set up to test pamac.

Didn’t mention paru, as it seems like a good idea to stick with one AUR helper as long as you can, because of the way they use their caches. When you do ‘pikaur -Sua’, it offers to upgrade to new versions of what it previously installed itself. If you switch to a new AUR helper, it doesn’t know what your old one installed.

Neither of you mentioned pip, but for me that’s a big part of it.
(One person told me we should use only pip and not AUR, so you can’t please everyone.)
I have about 500 files in my site-packages dirs, many from PyPI, so the methods are for dealing with that.

Mainly I wanted to help people with the Python part, and I’m not able to write things the way you’re expecting. I’m not a young guy, so don’t assume I’ve got time or inclination to go deeper into the low-level stuff… :slight_smile:

1 Like

pamac has problems with aur and cache, yes.

paru was just one other example of an aur helper, that uses a clean chroot.
https://wiki.archlinux.org/title/AUR_helpers

Yeah its a good idea to wipe out the cache of the previous one and load up all the foreign packages in the new one, ex:

yay -Scc
paru -Sua $(pacman -Qmq)

And of course theres the rebuild option for a cache if it already exists,ex:

paru -Sua $(pacman -Qmq) --rebuild

I mentioned this guide is targeting people with broken or old python …
so using python package manager may not work well.
Not to mention that your system package manager … you probably want from your OS Repositories rather than some other source.


The way to handle these updates are documented in various places.

This is one thats been attached to recent update announcements:

:information_source: You will need to rebuild any AUR Python packages that install files to site-packages or link to libpython3.10.so. :information_source:

Print a list of of packages that have files in /usr/lib/python3.10/ :

pacman -Qoq /usr/lib/python3.10/

Rebuild them all at once:*

pamac build $(pacman -Qoq /usr/lib/python3.10)
  • Note that if any fail to build, you’ll have to rebuild what’s remaining one or a few at a time.

Use rebuild-detector to see if anything else needs to be rebuilt:

checkrebuild

Of course you can then insert any relatively sane aur helper to do the same:

sudo pacman -Syu
paru -Sua $(pacman -Qoq /usr/lib/python3.10/) --rebuild

Neither do I, and only Unstable :slight_smile:

Not quite right.
Python is neither broken nor old: it’s new, and has moved to a new diectory.
Packages that were not originally installed from system repos
have been left behind in the old directory.

The method I gave for re-installing pikaur works fine.
lectrode’s method (git clone etc) looks quicker, but I didn’t know about it.
Also it assumes they have git installed, and maybe some noobs don’t…

Not sure what that means.
There’s no AUR helper in the system repos now (yay was removed).

The sequence of events is :

  1. packages from system repos were already re-installed by system update;
  2. my guide first re-installs AUR packages;
  3. then it gets pip to re-install whatever is left.

So this one I wrote can take a back seat, and maybe
people who are interested in Python will read it.

I’m a bit out of my depth, not quite expert enough,
but the guide does work, I think. :slight_smile:

Pamac has AUR functionality. Also, yay was re-introduced, so both are able to build and install AUR packages.

Hi,
first time post :slight_smile:

I just did the long version for the current update and nearly deleted all packages.

Reason: after full upgrade + reboot, my system did not have following folder “/usr/lib/python3.${OLD}”

echo "$OLD/$NEW"
10/11
pacman -Qnq $(pacman -Qoq /usr/lib/python3.${OLD}/site-packages) > SYS_pkgs_failed.txt
error: No package owns /usr/lib/python3.10/site-packages
wc -l SYS_pkgs_failed.txt
1529 SYS_pkgs_failed.txt

Because of the failed pacman -Qoq all 1529 packages where listed in the SYS_pkgs_failed.txt file.

Maybe you could mention this as warning above in the OP or build a error handling around the statement.

PS: Thank you for the guide :wink:

1 Like

Wow, very surprising. Thanks. I will edit.
I discovered recently that the old site-packages could be absent,
but expected that the result would be an empty output file.