Not entirely … pip list without any flags acts as if it is sudo or global pip.
Which is why the list action above should invoke the --user flag.
Though … it should probably get rid of the path portion as it should not be required … and there is the slight chance that the location is incorrect, whether due to configuration or otherwise.
and then gets a permission error.
I’ve put “–user” back into that last ‘pip install’ command.
(I hadn’t taken it out of the other one.)
Hang on, you said ‘pip list’ …
It works right for me as it is, but not with --user :
$ pip list --user --path $HOME/.local/lib/python3.9/site-packages/
ERROR: Cannot combine '--path' with '--user' or '--local'
As I said … the path should not be there.
Simply …
pip list
will return as if it is sudo or global.
pip list --user
will return correctly --user installed pip packages, whatever the actual configured path.
After the upgrade it is required.
I simplified the procedure by cutting out the need to do anything before the upgrade.
But for the current version.
I’m telling them to list the old version,
after they’ve done the upgrade.
oh right … afterwards it makes sense.
But you can trim it slightly …
--path $HOME/.local/lib/python3.9/
[jh@SSD2 220106 17:06:26 zp0]$ pip list --path $HOME/.local/lib/python3.9/site-packages/Package Version
------------------ -------
commonmark 0.9.1
filetype 1.0.7
font-line 3.1.1
fonttools 4.11.0
gdown 3.12.2
Genshi 0.7.5
html2docx 1.3.1
mpv 0.1
pymdown-extensions 8.1.1
PySocks 1.7.1
python-docx 0.8.10
python-uinput 0.11.2
rst2html5 2.0
tqdm 4.57.0
xxhash 2.0.2
[jh@SSD2 220106 17:06:33 zp0]$ pip list --path $HOME/.local/lib/python3.9/
[jh@SSD2 220106 17:06:38 zp0]$
(nothing)
Odd. works here.
…oh … probably wildcard expand.
$ pip list --path /home/kropo/.local/lib/python3.10/*
Package Version
------- -------
cowsay 4.0
But … at that point its probably better to just leave it instead of using *
Thanks.
I’m hoping it’s good enough now to leave as is for a while …
Why oh why are you recommending such opinionated stance? 
There are reasons why someone installed those packages per user level instead of global level.
Please omit this recommendation and strive to preserve the old situation.
As in user level desired package should be installed per user level, global desired packages should be installed at global level.
For the rest, great tutorial 
It’s not really my opinion.
It’s in line with what seems to be general moderator policy
of telling people to do the safest thing.
It’s about official repos being safer than PyPI or the AUR.
And AUR being easier to upgrade than PyPI.
It’s easy to install something with pip and not know that it’s in the repos.
This advice is intended to get people familiar with the idea of checking the repos first.
The info is intended for relative newcomers.
Those who know enough, know that they can do things their own way. 
It’s common in Windows, but in Linux I haven’t noticed
anything about per-user app installs (expect with pip).
Can it be done with pacman, with the --root option?
Thank you for this “tutorial”. It finally solved a few issues I had with Qtile and some extensions, as everything is written and configured in Python. I didn’t install all old packages and that caused problems. Also great to see that native pamac is used, instead of a third party program yay.
thanks for the tutorial!
supposing that I reinstalled all my Python packages via pacman, would there be any point if I deleted the contents of $HOME/.local/lib/python3.9/site-packages, now that we’re on 3.10?
[I’m assuming “point” means “problem”.]
No, no problem.
You can remove all of the ‘$HOME/.local/lib/python3.9/’ tree.
Also, my machine has no ‘/usr/lib/python3.9/’ tree now.
I vaguely remember removing it myself.
Maybe it would have been removed by the upgrade,
but there was a package in ‘/usr/lib/python3.9/site-packages/’
that the system had not installed, so it left it there.
Use the following commands to see what you’ve got :
ls -l /usr/bin/python*
ls -ld /usr/lib/python*
If there’s no python3.9 in /usr/bin/, there’s no point having python3.9 in /usr/lib/.
But if you want to be cautious, just rename it first,
then if something complains you can rename it back.
Btw, I thought it was obvious from the background info
that you could remove the old version, but I suppose
not everyone has time to think through it all… 
After a Python upgrade, Python and pip use only the new one, not the old one.
With “opinionated” I mean the terminology used in software development.
In short, the tutorial gives little room for alternative ways of handling Python packages which fit better to the person’s workflow.
You barely want previous per-user installed Python packages installed as system-wide packages. Especially when you’ve a multiuser system where each user prefer a different version of a Python package. Or an user is not allowed to use specific software.
Not every Python package is provided in the Manjaro repos nor in the AUR. Or do not immediately up-to-date when upstream has a new version.
For some packages like youtube-dl, gallery-dl and PixivUtil2, it is as good as required to upgrade the packages with pip (or a different package manager like asdf). Because upstream delivers that many frequent new releases that Manjaro repos nor AUR can not caught up the update speed.
I agree with you on every point.
Later, I decided that for myself, I prefer pip/PyPI (with ‘--user’) rather than AUR.
Upgrading is actually easier than with AUR.
After you first wrote about it, some of my edits lifted PyPI to equal level with AUR:
If a package is available from both AUR and PyPI,
consider the quality of that package at each place …
Evaluate packages carefully on AUR and PyPI,
as malware has been found on both systems.
When I first wrote the guide, I was new to the idea of using the AUR for Python packages, and simply followed recommendations about that from experienced Arch/Manjaro users. Now I prefer PyPI for most packages.
To accomodate all your points, the whole thing would have to be re-written, or replaced, and I don’t really have time now.
A lot of users who know nothing about Python installations, versions, or package updating, were getting confused when Python did a major version bump.
All I tried to do was make a simple step-by-step guide for those users,
and I didn’t expect anyone else would look at it.
People with more complex needs would know better than to rely on this little guide, wouldn’t they?
You don’t have to edit the whole thing, just add a link to @RoestVrijStaal’s message to the end of your first post so people could follow it and learn more.
I’ve added it at the top,
along with a note about preferring pip over AUR
(the advantages of pip were already mentioned in the long version).
Last Christmas I tried to write this post in the wiki,
but it disappeared, so I wrote it here instead.
Today I discovered that my post is in the wiki, though now out of date,
so I’ll have to do some clean up when I have time …
This info was incredibly helpful to me, but I wanted something slightly more automated, so I made a bit of a python/shell scripting, since I’m pretty novice to shell and couldn’t write it all in shell. (Note: this is for updating packages via PyPi only)
This is my pip-update.sh :
#! /bin/bash
# Change python version below from 3.10 to whatever your current version is
pip list --path $HOME/.local/lib/python3.10/site-packages/ > .pip-lst.txt
python3 Pip-update.py .pip-lst.txt
tr '\n' '\0' < .pip-adj.txt | xargs -r -0 pip install -U
rm -f .pip-lst.txt .pip-adj.txt
The output from pip list is manipulated via a python script that makes a new file with just the package names, one per line. The script then takes the lines and feeds them individually to pip install -U.
# Pip-update.py
import sys
file = open(sys.argv[1],'r')
vfile = []
for i in file:
vfile.append(i)
file.close()
vfile = vfile[2:]
vfile_adj = []
for i in vfile:
index = 0
while (i[index] != ' '):
index = index + 1
vfile_adj.append(i[:index])
new_file = open('.pip-adj.txt','w')
for i in vfile_adj:
new_file.writelines(i)
new_file.writelines('\n')
new_file.close()
After saving both files, just run
sh pip-update.sh
Or whatever you named the shell script. Remember to rename the python script inside pip-update.sh in case of changes.
The only problem is that I couldn’t automate the picking of pathname given the current python version in the system, so it needs changing, e.g, from
$HOME/.local/lib/python3.10/site-packages/
To
$HOME/.local/lib/python3.11/site-packages/
When python 3.11 comes out. Of course, it shouldn’t be too hard to get this part of the code done, or to port it all to shell. But just in case someone finds it useful, I decided to post the code anyway.