How to safely compile locally different versions of linux commands?

Because /usr/local/bin is searched before /usr/bin - and the first match is what you get when you just use the name.
When you use the full path, you can choose which one to use.
(requires that you even know that you now have two and can choose between them)

Search order is defined in /etc/login.defs
and
/etc/profile
/etc/profile.d/*

echo $PATH gives the current one

2 Likes

I believe you can ignore systemd-path since you are just running the command with your user’s shell. After all the things systemd-path can do, the end result is still the PATH environment variable for your shell. So it is just going down the list in: echo $PATH, starting with the first directory.

From what I can tell, Manjaro adds this, on login, in: /etc/profile:

    14  # Append our default paths
    15  append_path '/usr/local/sbin'
    16  append_path '/usr/local/bin'
    17  append_path '/usr/bin'

With /usr/local/sbin getting appended first.

I try to leave these files alone, and it makes dealing with pacnews easier. So instead of modifying that file, I would place PATH=/usr/local/sbin:$PATH in ~/.bashrc or whatever shell you are using to only change it for your user.

1 Like

“flex” use KDE, and with kde, original order is set in sddm.conf (nice, today it’s the same list as profile)

[Users]
DefaultPath=/usr/local/sbin:/usr/local/bin:/usr/bin

Never compile software as root. Never.

Why not pick a folder in your Home?

For tests or if you the only user on the system to use this program, don’t install it system-wide. Just pick a folder in your Home. You can add it to your PATH, but this is only for your convenience and not necessary and will override the distro version unless you manually change the name.

Personally, I never install it, if it is just for a test. The make install command just copies files and creates folders. And this is the reason why this command needs sudo if you want to install a program system-wide for every user on the system. But if you the only user and you want to keep using the distro version, why install it in the first place.

The binary can be startet for the compile folder, it doesn’t need to be installed. The location of the binary differs between projects, for parted, the binary is in the parted folder within the folder the git clone command created (from your expmple in your first post /usr/local/src/parted/parted/parted).

5 Likes

As others have tried to explain already — but in such a way that it can easily be mistaken — the rationale is as follows…

  • Unlike in Microsoft Windows, UNIX systems only allow calling an executable file by its name as a command that is to be executed if this file is listed in one of the directories of your $PATH, or otherwise put, the current working directory is not part of the search path for executable files. This $PATH is a list of directories, with the listed and distinct search paths separated by a colon (“:”).

  • When you issue a command at the shell prompt, the shell will first look for whether the command is internal to the shell itself — even if an executable by the same name exists in a directory of the $PATH — and if this is not the case, then it will traverse the list of directories in the $PATH, whereby the first found instance of an executable file with the name of the command is used, provided that one has the permission to execute it.

  • It is always possible to invoke an executable file residing in a directory that’s not included in the $PATH, by providing an absolute or relative path to where the file is located. An absolute path is the path to the executable file from the top of the directory hierarchy (e.g. /usr/local/sbin/parted), whereas a relative path is the path to this executable file relative to the current working directory (e.g. ./scripts/somescript.sh).

  • There is no need to give your locally compiled parted a different name. Either invoke it via its absolute path, or create an alias that invokes it via its absolute path, but then it is best to give the alias itself a different name, so as to not have its existence prevent you from using the system-default version of the executable.

  • UNIX is a multi-user architecture, and so it is possible to set a different value for the $PATH in your own account versus the system-wide default $PATH, commonly via your ~/.bash_profile or the equivalent zsh environment file — possibly ~/.zprofile, but I use bash, so I don’t have any real experience with zsh. Command aliases and functions should go into your ~/.bashrc or ~/.zshrc, or in a third file that you source from within ~/.bashrc or ~/.zshrc.


Actually, no, that would be a bad idea.

  • /opt is for applications or tools which are installed from their vendors in a binary form, but outside of the scope of the package manager — e.g. if you install a software package in binary form straight from the vendor’s website, usually by way of a self-extracting archive or a script.

  • /usr/local is for software that was compiled from sources on the local machine, and/or for locally shared read-only data.

However, with regard to the latter, as @xabbu said… :point_down:

Exactly. Unpack the source code in your home directory somewhere, compile it under your own account, and then install it, either with sudo or as root.

5 Likes

Thank you everyone for your helpful insights.

For any other noobs like me reading this thread to clarify the steps from my first post.

Going forward I will compile code (e.g parted) into a dedicated ‘apps’ directory in my home dir, and not as root, like this…

cd /home/flex/apps/src
git clone https://git.savannah.gnu.org/git/parted.git
cd parted/
./bootstrap
./configure --prefix=/home/flex/apps --disable-static --disable-device-mapper
make

In my case I would aim to run the binary for testing purposes so would probably just run the binaries from where ever they get created.

Alternatively, to fully install I would still choose to have them install in to my home directory as follows which achieves that goal because of ./configure --prefix=/home/flex/apps above.

make install

I guess the binary will get “installed” into something like…
/home/flex/apps/[bin or sbin]/[name_of_app_binary]

Now since my aim here is for testing only and I want to ensure no overlap of an app I am testing that is also a system app I would do these steps, even though as explained above they are not strictly necessary.

  1. Make sure the installed dir is NOT in my PATH. ($echo $PATH)
  2. Rename the locally compiled application (e.g: parted → parted-v3.66)
  3. Create an alias in my ~/.bashrc to the re-named app so I can access it from anywhere and not have to specify the full path to it.
    alias parted-v3.66="/home/flex/apps/sbin/parted-v3.66"

And Finally
To satisfy my OCD: just to go back to a question I asked above that probably came across confusing.
Firstly, I understand that /usr/sbin is symlinked to /usr/bin.

This is the first part of my $PATH:
/home/flex/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin

With these…

/usr/local/sbin/parted (v3.66)
/usr/sbin/parted (v3.6)

If I do…
$parted -v it sees (v3.66) which makes sense.

Then if I do this…

sudo mv /usr/local/sbin/parted /usr/local/sbin/parted-patched
$parted -v

What happens? I would expect it to see v3.6 now because the only parted that exists on my system is in /usr/sbin which is v3.6.

Instead within the same bash shell Konsole window/tab I get this:
bash: /usr/local/sbin/parted: No such file or directory

If I open a new bash shell window in Konsole it then works as expected and $parted -v (sees v3.6)

But again if from this new shell window if I then do this:

sudo mv /usr/local/sbin/parted-patched /usr/local/sbin/parted
parted -v

I would expect it to see v3.66 since /usr/local/sbin/parted is v3.66 and it exists now again in the file system. Instead parted -v still sees v3.6.

I guess I stumbled upon behavior of Konsole or maybe bash or something where it caches into memory the state of the file system when you initially open a command line screen? You can imagine how confusing this past few days were for me trying to process a lot of new concepts and stuff that seemed to contradict everything!

Thank you,

Flex

1 Like

the declaration of the --prefix=/xyz in the configure step
configures the build and determines what will happen later, when the thing is built and then installed with
make install.

That is the only step where you might need sudo (admin rights) - when it is to be installed into a directory your normal user has got no writing rights.

That is where it will be installed.
Not where you compile it.

You can build it anywhere. Doesn’t matter.
--prefix=/xyz will later install it into /xyz

./configure --help
usually tells you all about it and much more

2 Likes

The command: which <COMMAND> will tell you the full path of any executable, before you attempt to run it. Even though you should be essentially getting the same results with your test. This way works a little better; for example, if you copied/moved/linked the wrong binary. Plus, it can be used for many other purposes.

I don’t use KDE, but in all my experience with PATHs, modifying it is instant. Like you have echo $PATH as you want, then run the command. You still have to reload your shell?

only
which -a COMMAND
will tell you about more than the first match in your $PATH if more than one with the same name exist

1 Like

@Nachlese Thanks for that command, really useful to know.

@Molski It’s frustrating but yeah as described in my last post there are scenarios where I need to reload my shell. I shouldn’t have to do that, it’s like the shell is unable to keep track of changes to the file system in real time such as mv’ing a binary.

Flex

When you make changes in the profile or shell startup files, yes you need to reload your shell. Like many applications, you have to restart after configuration file changes.

But you can test what to set it as just by typing PATH= /usr/local/... first.

@Molski Did you read my post above in the “And Finally” section?

I wasn’t making changes to my profile or shell startup files. The only changes I was making was renaming the parted binary using the mv command.

I found that after doing that I had to open a new shell tab in Konsole or else $parted -v was giving me the wrong information back.

Flex

That’s because you assigned a filename as a value to a variable and then changed the filename without also changing the value of the variable.

It’s a bug in the biological unit between the keyboard and the chair, not in the shell. :smirk:

1 Like

How should I do it then instead of:

sudo mv /usr/local/sbin/parted-patched /usr/local/sbin/parted

So I don’t have to reload the shell?

Flex

Reassign the variable. :man_shrugging:

@Aragorn Despite your best efforts at trying to help me I’m afraid in this case:

  1. I don’t know which is the filename (/usr/local/sbin/parted ?)
  2. Which is the variable (/usr/local/sbin/parted-patched ?)
  3. I don’t know how to “reassign the variable”.
  4. I don’t understand why I would need to anything more than just use the mv command and expect $parted -v to have an understanding of what has changed.

If you prefer I can start another thread elsewhere.

Flex

Yes.

You had me confused there because you were referring to an alias, but your pasted shell output suggests that you invoke it via the variable $parted.

At this point, I have no idea how you invoke it, but if it is indeed a variable, then you must assign the new value to it… :point_down:

name-of-variable="new-value"

If it is an alias, then you must redefine the alias… :point_down:

alias name-of-alias="the-command-you-wish-to-invoke"

There you go again. $parted is a variable, not an alias.

Ok, let me try to explain myself as clearly as I can.

Here are some commands I have just tried from a BASH shell in my Manjaro PC Konsole:

You see what is in my PATH and that /usr/local/sbin comes before /usr/bin and that I have two binaries called parted each in two different locations.

  • /usr/local/sbin/parted is version 3.66
  • /usr/bin/parted is version 3.6.
[flex@thinkpad ~]$ echo $PATH
/home/flex/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/lib/rustup/bin:/var/lib/snapd/snap/bin
[flex@thinkpad ~]$ which parted -a
/usr/local/sbin/parted
/usr/bin/parted
[flex@thinkpad ~]$ /usr/local/sbin/parted -v
parted (GNU parted) 3.6.6-c5bd-dirty
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
[flex@thinkpad ~]$ /usr/bin/parted -v
parted (GNU parted) 3.6
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
[flex@thinkpad ~]$ parted -v
parted (GNU parted) 3.6.6-c5bd-dirty
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
[flex@thinkpad ~]$ 
[flex@thinkpad ~]$ sudo mv /usr/local/sbin/parted /usr/local/sbin/parted-patched
[flex@thinkpad ~]$ 
[flex@thinkpad ~]$ which parted -a
/usr/bin/parted
[flex@thinkpad ~]$ parted -v
bash: /usr/local/sbin/parted: No such file or directory
[flex@thinkpad ~]$

I use the mv command to rename the parted binary in /usr/local/sbin to parted-patched. So I now only have one binary in my system called parted and that is the v3.6 parted in /usr/bin as confirmed by the command which parted -a.

My question:
At the end when I do $ parted -v why does it not see parted v3.6 but instead it says:
“bash: /usr/local/sbin/parted: No such file or directory”?

If I open a new shell, a new tab in Konsole, then parted -v does see the correct v3.6

Flex

Because your $PATH is a variable, and that variable is only read when you open a login shell, although what you say leads me to suspect that — at least on your system — it is also being read when starting an interactive shell. Maybe you have your $PATH defined in ~/.bashrc — normally it would in be ~/.bash_profile only.

Either way, the location of some of the commands in the $PATH is hashed, meaning that their location is read into a cache in memory, so that the shell doesn’t need to effectively scan each of the directories of the $PATH. Presumably that is what’s happening here, i.e. you changed the name of the executable, but it doesn’t correspond anymore to what’s in the cache.

1 Like

Did you create an alias?

alias | grep parted

command parted -v

If you create an alias in a shell it’ll work in that shell, when you open a new shell it doesn’t exist, unless you put it in your config (~/.bashrc or similar) first. :man_shrugging:

2 Likes