Problems with python modules

Hi all and thank you for any help you provide :slight_smile:

I recently installed manjaro again after quite some time. I had to move to windows to pursue a career in development and I’m happy to be back in a free operating system.

I’m having some difficulty getting the python module “requests” to work.
In any IDE, (Thonny, pycharm, IDLE, Spyder, Ninja I tried them all) I can’t get it to recognize that this module is actually installed. However, when I test it, it is indeed actually there when I use the terminal.
I tried reinstalling this module in the software center, and I was greeted with:

manjaro-system needs to be removed but it is a locked package - pacman needs to be removed but it is a locked package

Im not sure why this is attempting to remove manjaro-system and pacman to reinstall requests. But okay.
I tried using pip to uninstall and reinstall. Pip uninstalled, but wouldn’t reinstall. Unfortunately I lost the terminal output. And I don’t think it actually removed anything, because when I do pip -list it will show requests.
I have no issue with any other module installed on the system so far. I just would like it to work in an IDE I use.
Thank you for the help :slight_smile:

pip uninstall requests
sudo pacman -S python-requests

This suggests that something about the original installation of ‘requests’ was not right.

How did you install it?
Your best hope is to reverse that process.

If you first installed it with pip, then ‘pip uninstall requests’ should work.
If you used sudo when installing, you need to use it when uninstalling:
sudo pip uninstall requests’.

If you installed via pip, pamac cannot uninstall it, afaik.
(I guess “software center” is the pamac gui; I’ve never used it.)

If you manage to uninstall it successfully,
then try @ishaanbhimwal’s suggestion to install it again:
sudo pacman -S python-requests

It actually came default with the OS.
I successfully uninstalled requests with pip. As shown by this output:
sudo pip uninstall requests  :heavy_check_mark:
[sudo] password for loganh:

Uninstalling requests-2.26.0:
  Would remove:
    /usr/lib/python3.10/site-packages/requests
    /usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info
Proceed (y/n)? y
  Successfully uninstalled requests-2.26.0

however upon trying Sudo pacman -S python-requests I get this output:

resolving dependencies...
looking for conflicting packages...

Packages (1) python-requests-2.26.0-5

Total Installed Size:  0.45 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 python-requests-2.26.0-5-any is up to date
(1/1) checking keys in keyring                                                                                                                [#######################################################################################] 100%
(1/1) checking package integrity                                                                                                              [#######################################################################################] 100%
(1/1) loading package files                                                                                                                   [#######################################################################################] 100%
(1/1) checking for file conflicts                                                                                                             [#######################################################################################] 100%
(1/1) checking available disk space                                                                                                           [#######################################################################################] 100%
warning: could not get file information for usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info/
warning: could not get file information for usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info/PKG-INFO
warning: could not get file information for usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info/SOURCES.txt
warning: could not get file information for usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info/dependency_links.txt
warning: could not get file information for usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info/not-zip-safe
warning: could not get file information for usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info/requires.txt
warning: could not get file information for usr/lib/python3.10/site-packages/requests-2.26.0-py3.10.egg-info/top_level.txt
warning: could not get file information for usr/lib/python3.10/site-packages/requests/
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__init__.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/__init__.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/__init__.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/__version__.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/__version__.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/_internal_utils.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/_internal_utils.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/adapters.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/adapters.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/api.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/api.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/auth.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/auth.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/certs.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/certs.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/compat.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/compat.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/cookies.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/cookies.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/exceptions.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/exceptions.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/help.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/help.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/hooks.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/hooks.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/models.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/models.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/packages.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/packages.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/sessions.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/sessions.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/status_codes.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/status_codes.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/structures.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/structures.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/utils.cpython-310.opt-1.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__pycache__/utils.cpython-310.pyc
warning: could not get file information for usr/lib/python3.10/site-packages/requests/__version__.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/_internal_utils.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/adapters.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/api.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/auth.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/certs.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/compat.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/cookies.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/exceptions.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/help.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/hooks.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/models.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/packages.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/sessions.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/status_codes.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/structures.py
warning: could not get file information for usr/lib/python3.10/site-packages/requests/utils.py
:: Running pre-transaction hooks...
(1/1) Creating Timeshift snapshot before upgrade...
==> skipping timeshift-autosnap due skipRsyncAutosnap in /etc/timeshift-autosnap.conf set to TRUE.
:: Processing package changes...
(1/1) reinstalling python-requests                                                                                                            [#######################################################################################] 100%
:: Running post-transaction hooks...
(1/2) Arming ConditionNeedsUpdate...
(2/2) Refreshing PackageKit...

Don’t remove something with pip if you didn’t install it with pip.

I guess all of pacman’s “warning: could not get file information”
happened because you had removed it with pip.

Anyway, it looks like the pacman re-install succeeded.

Has it solved your original problem?

Perhaps not, as we haven’t really changed anything.

Two really strange things happened, and we have no idea why:

  1. In any IDE … I can’t get it to recognize that this module is actually installed.
    However, … it is indeed actually there when I use the terminal.

  2. I tried reinstalling this module in the software center, and I was greeted with:
    manjaro-system needs to be removed … - pacman needs to be removed …

EDIT:
In the post that Tomek mentioned, you might find the ‘Background info’ part is helpful.

Maybe “python-requests” is a dependency of the system, and the method of uninstalling tried to also uninstall everything that depends on “python-requests”.

Do what now? That’s an invalid command. Please check your spelling and syntax.

Tip: When pasting terminal output on Discourse forums, one can either…

  • Use the Preformatted text </> toolbar button–NOT the Quote " button.

  • Add three backticks ` above and below the text (Markdown):

    ```
    type or paste code here
    ```

  • Use HTML:

    <pre><code>
    type or paste code here
    </pre></code>

Please edit your post accordingly.

Check sys.path (a) in the IDEs and (b) with python in terminal.
Is “requests” located in some place that’s not in the IDE’s path?

Does it need to be imported a different way?
Like ‘from some_package import requests’ ?

Hi,
Upon searching sys.path in the terminal I found:

>>> sys.path
['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/home/loganh/.local/lib/python3.10/site-packages', '/usr/lib/python3.10/site-packages']

Upon sys.path in pycharm:

['/home/loganh/PycharmProjects/The Great Beans', '/home/loganh/PycharmProjects/The Great Beans', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/home/loganh/PycharmProjects/Learn Python/The Great Beans/lib/python3.10/site-packages']

(The project name is The Great Beans)
Requests is found in:
/lib/python3.10/site-packages/jedi/third_party/typeshed/third_party/2and3/requests

Also, Upon recent use of pip ever since I uninstalled requests with it, I get this error every time I try to run it:

 pip list                                                                                                                                                                                                                   1 ✘ 
Traceback (most recent call last):
  File "/usr/bin/pip", line 33, in <module>
    sys.exit(load_entry_point('pip==20.3.4', 'console_scripts', 'pip')())
  File "/usr/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/usr/lib/python3.10/site-packages/pip/_internal/commands/__init__.py", line 105, in create_command
    module = importlib.import_module(module_path)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3.10/site-packages/pip/_internal/commands/list.py", line 9, in <module>
    from pip._internal.cli.req_command import IndexGroupCommand
  File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 18, in <module>
    from pip._internal.index.collector import LinkCollector
  File "/usr/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 14, in <module>
    from pip._vendor import html5lib, requests
ImportError: cannot import name 'requests' from 'pip._vendor' (/usr/lib/python3.10/site-packages/pip/_vendor/__init__.py)

Thank you for the post.
That was a typo I made, I did not actually paste that into the terminal. Thanks for letting me know about highlighting output :slight_smile: I am new here so my apologies.

EDIT at UTC 2022-01-26 02:50 :
This post and my next few are quite useless. Please jump to post 19.

That’s quite normal.

EDIT 22-01-26: I guess “PycharmProjects/Learn Python/The Great Beans” is like a virtual env.

EDIT 22-01-26: I have that too, but don’t know what installed it.
I also have requests at ‘/usr/lib/python3.10/site-packages/requests/’,
and this would be the one that you removed with pip.

The start of the last path is not complete.
Was it ‘/usr/lib/…’, ‘/home/loganh/lib/…’, or somewhere else?

Anyway, if you succesfully installed ‘requests’ with ‘pacman’,
it should now be in ‘/usr/lib/python3.10/site-packages/’
and available to import from anywhere.

But the other installations of ‘requests’ may will take precedence (or conflict?)
when you’re in those frameworks that also installed it.

Hopefully someone who’s familiar with those IDEs and frameworks
will give you advice that’s more specific…

googling for “pip._vendor” “requests” led to this:
https://stackoverflow.com/a/70431656/3147828
I have no idea if it will solve your problem.

EDIT: that solution was for Ubuntu.
On Arch/Mamjaro we should do ‘sudo pacman -S python-pip’.
Seems like the problem is mainly that pip’s cache doesn’t have what it expects.
Re-installing python-pip with pacman won’t do any harm, and might fix its cache.

Not in Python I guess – you seem to be fairly new to python … ?

Not only free, but quite different.
Not sure if your Windows-style approach of installing multiple IDEs
is really suited to Unix or Python – it seems to have caused a tangle already.

I get the impression that you thought Linux and Python could handle being thrown around by installing many IDEs (and maybe frameworks too?).

Maybe it would be better to first study what you’re getting into,
and when you’ve decided which IDE/framework you want to try,
install it carefully.

The basics are covered by the Python Docs : Using Python on Unix platforms

You may want to study how things like Anaconda interact (or don’t) with system python.
(Spyder is often installed as part of Anaconda, so you might have it.)

Can’t tell you much about complex IDEs.
15 years ago I learned to use SciTE (a text editor that can run code), and I still use it.
My projects are small and I work alone, so I don’t need a complex IDE.

Have used Wing-IDE Personal a bit.
It’s quite easy to understand, and helped with debugging a few times.

The last path simply refers to the root directory I believe.

I have attempted reinstalling requests with pamac and it does it with no errors but doesn’t seem to want to work anyway.

Actually pursued one of JS and backend development. I am still learning python, yes.

I only installed multiple IDE’s to attempt to find a way to fix this problem on my own. I usually only use Pycharm as I learned how to use it fairly competently in Windows.

I faced with similar issue recently

On my system, ‘requests’ is found here:

/usr/lib/python3.10/site-packages/requests/
/usr/lib/python3.10/site-packages/jedi/third_party/typeshed/third_party/2and3/requests/
/usr/lib/python3.10/site-packages/jedi/third_party/django-stubs/django-stubs/contrib/sites/requests.pyi

The first line is the one that should be available everywhere.
(I didn’t knowingly install anything that pulls in ‘jedi’, so it might be part of the OS.)

Compare your sys.path results below, and you’ll notice that
when you’re working in ‘PycharmProjects’,
there’s no access to system python’s site-packages.
“PycharmProjects/Learn Python/The Great Beans”
seems to be like a virtual env that does not include system site-packages.

So even when ‘requests’ is properly installed in system site-packages,
you can’t import it while you’re in “The Great Beans”.

You would have to install ‘requests’ within that environment.
Does PyCharm have a way to do that?
(If not, maybe the way is ‘pip install --user’ within that env.)

sys.path

system python:

['',
'/usr/lib/python310.zip',
'/usr/lib/python3.10',
'/usr/lib/python3.10/lib-dynload',
'/home/loganh/.local/lib/python3.10/site-packages',
'/usr/lib/python3.10/site-packages']

pycharm:

['/home/loganh/PycharmProjects/The Great Beans',
'/home/loganh/PycharmProjects/The Great Beans',
'/usr/lib/python310.zip',
'/usr/lib/python3.10',
'/usr/lib/python3.10/lib-dynload',
'/home/loganh/PycharmProjects/Learn Python/The Great Beans/lib/python3.10/site-packages']

The other problems flow from the fact that you didn’t know the above point,
and tried re-installing system-level ‘requests’ instead.

attempted reinstalling requests … doesn’t seem to want to work anyway.

If you meant only “doesn’t seem to want to work in PyCharm”,
then the info above probably solves that problem…

Don’t know how to solve that problem, but here are some bits of related info.

  1. check the pip documentation

  2. inside the file /usr/lib/python3.10/site-packages/pip/_vendor/__init__.py,
    ‘requests’ is mentioned; seems this is where the apparent dependency comes from;
    I don’t know how it works.

  3. in my working system, info from pacman :
    ‘requests’ is not a dependency of ‘pip’ at this level.

    ‘pacman -Qo’ looks for owners.
    ‘pacman -Qd’ looks for dependencies.

    [jh@SSD2 220126 10:44:04 ~]$ pacman -Q python-pip
    python-pip 20.3.4-4
    
    [jh@SSD2 220126 10:44:49 ~]$ pacman -Q python-requests
    python-requests 2.26.0-5
    
    [jh@SSD2 220126 10:46:10 ~]$ pacman -Qo python-pip
    error: No package owns python-pip
    
    [jh@SSD2 220126 10:46:18 ~]$ pacman -Qd python-pip
    [nothing]
    
    [jh@SSD2 220126 10:45:02 ~]$ pacman -Qo python-requests
    error: No package owns python-requests
    
    [jh@SSD2 220126 10:45:53 ~]$ pacman -Qd python-requests
    python-requests 2.26.0-5
    
  4. in my working system, when imported in python,
    the versions of pip and requests match those in pacman,
    and the paths show that they are located in system python.

    >>> import pip
    >>> pip.__path__
    ['/usr/lib/python3.10/site-packages/pip']
    >>> pip.__file__
    '/usr/lib/python3.10/site-packages/pip/__init__.py'
    >>> pip.__version__
    '20.3.4'
    
    >>> import requests
    >>> requests.__path__
    ['/usr/lib/python3.10/site-packages/requests']
    >>> requests.__file__
    '/usr/lib/python3.10/site-packages/requests/__init__.py'
    >>> requests.__version__
    '2.26.0'