Call for testing: Making `pip` default to `--user`

Since time immemorial people have used sudo pip install to work around permissions issues rather than simply use pip install --user.

There are long, years-old discussions to address this upstream as well as other approaches to add a separate directory for modules installer with pip.

However, there appears to be an easier way to default to using --user.

To test this you will need updates to two packages, filesystem-2018.9-3 and python-pip-19.2.3-1.2 (both currently in unstable).

filesystem-2018.9-3 adds $HOME/.local/bin to your $PATH by default. python-pip-19.2.3-1.0 adds a global configuration file (/etc/pip.conf).

Combined, any run of pip install will now default to installing under your $HOME with binaries available under $HOME/.local/bin.

This should be transparent to most people and prevent all of the various "MY PYTHONS IS A CONFLICT!" threads. This won't fix things for people who have already run sudo pip install but they shouldn't have done that in the first place.

If you really want to avoid using --user you can run with

PIP_USER=false pip list

which ignores the default setting.

This is all working fine in my initial testing, but please test and feed back any issues.


This is:

  • Working without issue
  • Working but with issues (post)
  • Completely broken (post)

0 voters

4 Likes

I was not aware there was a standard on that.

I have for years used $HOME/bin and with python used the virtualenvwrapper to wrap local python.

I have wondered how the files placed in /etc/profile.d comes into play.

On my Openbox installs I have to source a file from the folder for it to be active - the profile.d indicates that some sort automation would be active but I never had any succes.


Just installed the update filesystem package - it works - so I am to speculate why I have to source other files.


Tested with pip - it works as announced - just need to ensure the ~/.local/bin is an existing folder.

I have adapted the Openbox config to include the folder.

If it's not an existing directory then it won't be added to your $PATH. If pip installs something into that directory then it will be created.

The only edge-case for a failure will be not having the directory, opening a shell and installing something using pip, then it won't be visible until you open a new shell.

:: Processing package changes...
( 1/11) upgrading filesystem                             [##############################] 100%
warning: directory permissions differ on /root/
filesystem: 755  package: 750

[this one above is kinda funny because of the coincidence of names and terms. heh.]

And also, after reboot I still do not see "HOME/.local/bin to your $PATH"

$ echo $PATH
/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

That looks a bit loose for the /root directory.

Do you have that directory? It won't be added to your $PATH if it doesn't exist.

funny enough, I dont. ..So, technically one of the steps would be 'create that directory'.

It should be created when you run pip install $SOMETHINGWITHANEXECUTABLE.

It made more sense to me to not include a non-existent directory in the $PATH... :man_shrugging:

I have never had any problems (but do not equate this with any thoughts that I know what I am doing), it is really easy to undo and it only tends to happens to me if an application has a new dependency.

I have several programs (e.g. autokey-py3) that I install from pip (avec sudo) and always have.

..oh..it was my understanding that it needed to be in PATH before pip install ... would honor that placements for all this voodoo to hang together...

hmm ... let me find something half innocuous, and remove that folder first ..

-EDIT-

Yes, @jonathon you are correct..
It gets added to PATH automatically if exists (after reinitialize, such as log out)
pip install creates the folders ~/.local/lib and ~/local/bin and places things there automatically.

but

If the folders didnt exist in the first place .. then you will still get something like this until PATH is regenerated somehow (such as restart):

 WARNING: The scripts fonttools, pyftmerge, pyftsubset and ttx are installed in '~/.local/bin' which is not on PATH.
 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
 WARNING: The script dehinter is installed in '~/.local/bin' which is not on PATH.
 Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Hmm.

I can hide that via pip.conf. Question is, is it better to hide it or leave it so people know they need to do something with their path?

Trouble is, if they then take the initiative and add the location to their PATH they'll end up with it added again later...

OR I just add $HOME/.local/bin to the PATH no matter if it exists yet

right .. its like there should be some sort of 'hook' in between those actions somewhere ..

I dont think the manual way is best, unless its the 'only way' - it shouldnt be redundant and warned about. Though the warning itself isnt the issue.

(i also assume there would be functional limitations of those installed things until PATH is regenerated too)

...

I tentatively find this best at the moment.

I can't think of a way of handling a "stateful" path entry.

It needs to be present in advance to prevent pip complaining, and having a non-existent entry won't cause any issues.

OK, I think I'll push a new filesystem package with that change.

Edit: filesystem-2018.9-3 inbound.

The timing of this is hilarious. I managed to trash my system because I didn't know what I was doing with pip. Slowly bringing it back into ship shape, but I'm a bit in over my head. :joy:

1 Like

The only problem with this is that its reenforcing bad habits. They’re not learning to do things correctly by stuffing their systems up and being forced to fix it.

Maybe have an alias/function for “sudo pip” in .bashrc that’d spit out an error with the correct syntax to use.

Something like this...

sudo () {
    if [[ -z "$@" ]]; then
        command sudo
    else
        if [[ "$@" = "pip "* ]]; then
            echo -e "\e[1m\e[31mERROR:\e[0m Running pip as root is not allowed!"
            echo -e "\e[1m\e[33mINFO:\e[0m  A correct usage example is 'pip install --user <package>'"
        else
            command sudo "$@"
        fi
    fi
}

Or use the above in conjuction with your solution, that way existing users will be saved from their own mistakes and newly installed users will be forced to use and learn the correct syntax from the get go. You could even change the error message to be something more generic like use pip without sudo.

Manjaros bash is out of date, might be a good time to do it :grin: Especially with 18.1 release coming soon. There will be ALOT of fresh installs happening.

Edit: Tweak function.

Overriding sudo just for pip seems a bit heavyweight.

Given the upstream discussions around making --user the default I'm happy that this is the correct approach.

Are there any consequences we need to consider, with AUR packages that may use pip during building/installing various type of packages that can exist in AUR?
Packages maybe installed before this change, as well.

I don't think so; packages shouldn't be using pip for anything as that's mixing package managers.

If you can point me to an example I can have a look.

Not sure, but there are problems with joplin which uses node.js.

Hmm... node.js is quite different to Python...

1 Like

I've just followed a link to this discussion from a pip issue.

When testing, please ensure that if you create a separate environment (with python3 -m venv, or virtualenvwrapper, or conda), and use pip install with that environment active, it installs packages into that environment. I.e. an active environment should override the global configuration telling it to use --user. If this doesn't work, Python environments are broken, which will cause massive confusion.

(I genuinely don't know whether it will work - I'm not familiar with the details of how pip loads config.)

1 Like

Forum kindly sponsored by Bytemark