PYTHON interpreter UPgrade: Poetry

Thanky a lot for the detailed and very well structured HowTo! I followed without an issue up to section D4 but there is a problem:

$ pip install --user -r pip_freeze_user.txt
ERROR: Cannot install -r pip_freeze_user.txt (line 45) and urllib3==2.0.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested urllib3==2.0.2
    poetry 1.5.1 depends on urllib3<2.0.0 and >=1.26.0
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

Removed the line and tried again, the issue is with lines 45…47:

  • poetry==1.5.1
  • poetry-core==1.6.1
  • poetry-plugin-export==1.4.0

Removed all three lines and:

$ pip install --user -r pip_freeze_user.txt
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
poetry 1.4.2 requires platformdirs<3.0.0,>=2.5.2, but you have platformdirs 3.5.0 which is incompatible.
poetry 1.4.2 requires urllib3<2.0.0,>=1.26.0, but you have urllib3 2.0.2 which is incompatible.

How to solve this? Thank you!

I’m not totally sure, as I never had to deal with this kind of situation before.

The guide says to just keep notes of what didn’t work and deal with them later.
In this case, when you move on to the pip upgrade page.

At the point where you got to, you are only re-installing pip packages,
and I guess the other packages that are too new
have just been installed from Manjaro or AUR.

So keep your list files, and see what happens when you go to the pip upgrade page.

Thank you,

But what do you mean with “pip upgrade page”?

Now I fired:

$ pip install --user --upgrade poetry
      Successfully uninstalled urllib3-2.0.2
Successfully installed poetry-1.5.1 poetry-core-1.6.1 poetry-plugin-export-1.4.0 urllib3-1.26.16

$ pip check
No broken requirements found.

Am I done or is anything left?

Keep on working through part D4, and you’ll come to the link very soon :slight_smile:

Great! :slight_smile:

Did you finish on the previous page? – part D5 and onwards.
Go back there if you didn’t.

OK, did it now as follows, finally an issue with pip check:

$ pip list --user --outdated --not-required > upgrade_list.txt
Package        Version Latest       Type
-------------- ------- ------------ -----
ipython        8.13.2  8.14.0       wheel
lit            16.0.3  16.0.5.post0 sdist
pycryptodomex  3.17    3.18.0       wheel
PyScreeze      0.1.28  0.1.29       sdist
PySide6-Addons 6.5.0      wheel
PySimpleGUI    4.60.4  4.60.5       wheel
pytweening     1.0.5   1.0.7        sdist
stevedore      5.0.0   5.1.0        wheel

$ pip install --user --upgrade ipython
    Uninstalling ipython-8.13.2:
      Successfully uninstalled ipython-8.13.2
Successfully installed ipython-8.14.0

$ pip install --user --upgrade lit
    Uninstalling lit-16.0.3:
      Successfully uninstalled lit-16.0.3
Successfully installed lit-16.0.5.post0

$ pip install --user --upgrade pycryptodomex
    Uninstalling pycryptodomex-3.17:
      Successfully uninstalled pycryptodomex-3.17
Successfully installed pycryptodomex-3.18.0

$ pip install --user --upgrade PyScreeze
    Uninstalling PyScreeze-0.1.28:
      Successfully uninstalled PyScreeze-0.1.28
Successfully installed EasyProcess-1.1 PyScreeze-0.1.29 entrypoint2-1.1 mss-9.0.1 pyscreenshot-3.1

$ pip install --user --upgrade PySide6-Addons
      Successfully uninstalled PySide6-Addons-6.5.0
Successfully installed PySide6-Addons- PySide6-Essentials- shiboken6-

$ pip install --user --upgrade PySimpleGUI
      Successfully uninstalled PySimpleGUI-4.60.4
Successfully installed PySimpleGUI-4.60.5

$ pip install --user --upgrade pytweening
      Successfully uninstalled pytweening-1.0.5
Successfully installed pytweening-1.0.7

$ pip install --user --upgrade stevedore
      Successfully uninstalled stevedore-5.0.0
Successfully installed stevedore-5.1.0

$ pip list --user --outdated --not-required

$ pip check
pyside6 6.5.0 has requirement shiboken6==6.5.0, but you have shiboken6
shiboken6-generator 6.5.0 has requirement shiboken6==6.5.0, but you have shiboken6

Mmm … can you not see it for yourself?
shiboken6 belongs with pyside6.
3 packages that work together, but 2 are on 6.5.0, one on …
They should all be on the same version, so make that happen.

Btw, I wonder how did they get to be different? !

Are some of them from system python, or AUR, and some from pip.
If so, that’s a bad scene. They should all be from same place.

Thank you, but both packages are from repository:

Now I have:

$ pip check
shiboken6-generator 6.5.0 has requirement shiboken6==6.5.0, but you have shiboken6
$ pamac remove shiboken6-generator
Error File not found: shiboken6-generator

No idea, what I should do …

Look at the version numbers on those images.
You did not see something very obvious.

Ask yourself:
Where is shiboken
Why does it exist on your system?
Does anything depend on it?
Can/should you uninstall it?

I wrote a long explanation of Python (part B) so that people can understand enough to sort things out for themselves.
I’m sorry, but it’s not practical for me to solve everyone’s specific problems.

I’ll give you a bit more clue.
Read about the 2 “site-packages” folders.
Be aware that pip uses one, and Manjaro/AUR use the other.

It looks like pyside6 and shiboken6, both, are both in the right place.
shiboken6 is in the other place, but should not be installed at all.

maybe pyside6 is also installed.
maybe you installed pyside6 with pip,
but maybe you shouldn’t because the system installs it.

but maybe it doesn’t matter.
maybe the system runs one version (6502) as root,
and some app you installed with pip runs the other (6511) in user-space,
and they don’t collide.
i don’t know.

I could not find shiboken Removed the pip-pacakages:

$ pip uninstall shiboken6-generator
$ pip uninstall shiboken6
$ pip uninstall pyside6-essentials

$ pip check
No broken requirements found. was probably the one you removed.

I’d be interested to know if your system and your apps work OK now.

Btw, do you really need all of the pip packages you had installed?
poetry is an alternative to pip.
PySimpleGUI is an alternative to pyside.
If you’re going to build an app with PySimpleGUI or pyside
you need a much better understanding of python installations…

you did.
you need to spend more time reading your command ouput :slight_smile:
this is in one of your messages:

$ pip install --user --upgrade PySide6-Addons
Successfully uninstalled PySide6-Addons-6.5.0
Successfully installed

be aware that pip list --user and pip install --user
do not look at system site-packages. but pip check does.
so maybe ‘pip check’ finds conflicts that don’t really matter,
coz system and user can run separate versions (i think).

(it’s late here, i have to stop now.)

Thanks a lot for your support!

Yes, everything works fine and

$ pip check
No broken requirements found

What is the goal?

Poetry is a replacement for pip that automatically creates a virtual environment for you, it is in the repos python-poetry no need to use pip to install it.

“maybe ‘pip check’ finds conflicts that don’t really matter”.
I think this is true.

When the system starts a program, it does not see your user site-packages, so there’s no conflict.

When you run a program as user, it imports from user site-packages first, and imports from system space only if what it wants is not in user space.
Conflict is possible, but you can avoid it by taking care about what you install.

I suspect that the OP had followed a guide intended for Windows or Mac users,
saying to install all those things with pip because there’s no system repo or AUR.

Thank you @j77h and @codesardine

The strange thing: Poetry came obviously with the last MANJARO-update and produced the error as shown in my first post here; I have a listings of installed packages almost monthly and poetry showe up yesterday for the first time. I Never used poetry …

And: Yes, I know I may have an issue with PYTHON-packages from repository and pip. I startet with PYTHON few months ago and installed for my applications several packages like pandas, numpy, gpx-cmd-tools,, gpxpy using pip. Meanwhile I learned, that it is recommended to use a Virtual Environment. But I have no idea how to do this and how to transfer the approx. 200 PYTHON-packages, distinguishing if I really need each or not.

But a Virtual Environment might not be necessary.

Packages installed into the user-space ‘site-packages’
(as with ‘pip install --user ...’) cannot affect system python,
because “system python” (i.e. a python process started by the system)
never imports packages from user-space,

The converse is not true though:
a python process started by a user
does import from system ‘site-packages’.

BUT, it looks in user ‘site-packages’ first
(user ‘site-packages’ is earlier in the import path)
and then looks in system ‘site-packages’ only if
the target package is not in user ‘site-packages’.

So your user-space pyside/shiboken 6.5.1
was NOT in conflict with system-space pyside/shiboken 6.5.0.
(But it’s not needed if your user-space packages
are happy to run on system-space pyside/shiboken.)

My view is that you don’t need a virtual environment
as long as the packages in user ‘site-packages’ can all
work with the same versions of common dependencies.

If you make a virtual environment with ‘venv’,
basically all it gives you is another separate user ‘site-packages’ folder.
It still runs the system python.

A bit more about PySide6 and double installations.

Probably I was wrong to tell you to remove your PySide6 that was installed by pip.
This one:

Today I noticed that PySide6 in system site-packages weighs about ~50 Mo
but the PySide6 in user site-packages weighs about ~230 Mo.
If in future you find that some app complains about
parts of PySide6 missing, this difference would explain why.

I think it would be safe to disregard the warning from ‘pip check’.
My own system had PySide6 installed in both places for a long time, and I never noticed.

You can re-install them with
pip install --user PySide6
This should pull in Essentials, Addons and shiboken as dependencies.
Probably it would get them out of cache and not do any download.

It might be best to do this as soon as you have time, rather than leaving it
until there’s an error, and hoping that you then remember this message.

If re-installing it does cause problems, you could easily do
pip uninstall PySide6 to remove them all again.
It tells you what it would uninstall from where, and asks for yes/no;
then you should say no if it would remove it from ‘/usr/lib/...’ ,
but say yes to remove it from ‘/home/your-name/.local/lib/...’.

EDIT: you could have the best of both worlds
(two installs of PySide6 but no conflict) by doing
pip install --user 'PySide6==6.5.0'
(or whatever the then-current version is for system PySide6)

Thanks a lot @j77h for your support.

I lost control after all my changes. Actually I did this now as you recommended:

$ pip install --user PySide6
Requirement already satisfied: PySide6 in /usr/lib/python3.11/site-packages (6.5.0)
Requirement already satisfied: shiboken6==6.5.0 in /usr/lib/python3.11/site-packages (from PySide6) (6.5.0)
$ pip check
No broken requirements found.

Seems to be OK, what do you think? Thanks a lot!!