How to safely compile locally different versions of linux commands?

[flexmcmurphy]

This question is related to another post in which I asked about the linux parted command.

The parted command did not have the functionality I wanted, someone proposed a patch to give this functionality. I downloaded the latest source code and compiled in the patch and it now does what I want. I followed instructions from here and here.

Briefly, these are the commands I ran:

cd /usr/local/src
sudo git clone Index of /git/parted.git/
cd parted/
sudo ./bootstrap
sudo ./configure --prefix=/usr/local --disable-static --disable-device-mapper
sudo make
sudo make install

This gave me a parted executable in /usr/local/sbin

Now from a command prompt, anywhere in my system, if I run this command:

[flex@thinkpad ~]$ parted -v
parted (GNU parted) 3.6.6-c5bd-dirty

I also still have the original parted in /usr/sbin…

[flex@thinkpad ~]$ /usr/sbin/parted_orig -v
parted (GNU parted) 3.6

My question:
What I really wanted was to keep the original parted (v3.6) for everyday use and only have this patched parted (v3.6.6-c5bd-dirty) to play with. I thought I would achieve that by specifying /usr/local in the ./configure command but my Manjaro system seems to now be using the patched version by default.

How do I tell Manjaro to use the original version? I’d like to learn about these amazing open source tools and C programming. But what is the best way to locally compile different versions of linux tools for testing and learning but without having my system use them by default? I just want these executables in a place separate from where Manjaro will look for them by default.

Cheers,

Flex

But when I specifically ask

[cscs]

@flexmcmurphy

What I really wanted was to keep the original parted (v3.6) for everyday use and only have this patched parted (v3.6.6-c5bd-dirty) to play with.

So rename the binary you manually created;

sudo mv /usr/local/sbin/parted /usr/local/sbin/parted-patched

Now use parted-patched when you want that.

Also note that of course this software will not be updated along with your system packages.
You must be responsible for that. Such as by performing the download/compile again when needed.

[Nachlese]

The original is in /usr/bin
parted consists of more than just the binary - there is also parts in /lib and more

You should be good when you call your version with the full PATH to it.
I don’t know much about aliases - this may work like renaming the executable.

[flexmcmurphy]

@cscs
Cheers but is there not anything else I would need to be careful about? Is it enough to just compile the code as I did, install in to /usr/local and give the executable a different name than the original “proper” version?

Like @Nachlese said… What if I found an interesting branch of parted on GitHub with some modifications that change libparted in a way that breaks my GParted which I think uses that library also?

Does compiling these tools like I did this time (in to /usr/local) really sandbox them away from their existing versions within my “main system”? I can see those libparted libraries in /usr/lib and also newer copies now in /usr/local/lib so probably the way I did it will keep my patched versions totally separate?

@cscs As you say maybe if I did mess up a shared system library then the next Manjaro update would probably fix that so my main system commands will work again (but break my patched versions). I would be ok with that as above all I don’t want to mess up my system.

Maybe it’s better I do this type of stuff in a VM for testing.

Cheers,

Flex

[papajoke]

@flexmcmurphy

without having my system use them by default? I just want these executables in a place separate from where Manjaro will look for them by default.

system and user, use $PATH, first found = first to run

 systemd-path | grep search-bin | sed 's/:/\n  /g'

and, you can use /opt/xxx/ for your own applications

[cscs]

@flexmcmurphy

sudo ./configure --prefix=/usr/local

Without actually inspecting anything I would think this means all of your library stuff from the compilation goes here in /usr/local as opposed to /usr (where the packaged files go).

As long as nothing is relying on that path being exactly as originally created (doubtful), then simply using a different name should be sufficient.

The alias approach mentioned should work too. In .bashrc for example;

alias parted-patched '/usr/local/sbin/parted'

Well - it would work for making parted-patched run that executable.
If while keeping the same name it is being sourced before the original then you may want to combine it with another alias;

alias parted '/usr/bin/parted'

PS.

I also noticed you stated

@flexmcmurphy

I also still have the original parted in /usr/sbin…

But that is incorrect.
Well - it should exist, but as a symlink to the real location, which is /usr/bin/parted.
(or rather /usr/sbin is symlinked to /usr/bin)

[Nachlese]

@flexmcmurphy

./configure --prefix=/usr/local

That is what /usr/local is for
or /opt …
To separate your own stuff from the base system.

[soundofthunder]

You could contact the maintainer of parted-git and see what interest (if any) they might have in including the patch. Nothing ventured, nothing gained. Of course the compulsory warning follows that the AUR is unsupported by Manjaro (or Arch); you’d still be effectively on your own.

[flexmcmurphy]

@soundofthunder I have let them know at my GNU Parted bug report but based on the response they gave me earlier I think they would prefer you only use parted in interactive mode anyway and not allow parted do any potentially destructive stuff in non-interactive (script) mode.

@papajoke Your command was really useful to understand the order in which Manjaro will search for binaries.

systemd-path | grep search-bin | sed 's/:/\n  /g'

Using this I can see on my system that (?strangely) Manjaro will look to /usr/local/bin first and then /usr/bin after that. I suppose that’s what “search-binaries-default” is about?

[flex@thinkpad bin]$ systemd-path | grep search-bin | sed ‘s/:/\n /g’
search-binaries
/home/flex/.local/bin
/usr/local/sbin
/usr/local/bin
/usr/bin
/usr/lib/jvm/default/bin
/usr/bin/site_perl
/usr/bin/vendor_perl
/usr/bin/core_perl
/usr/lib/rustup/bin
/var/lib/snapd/snap/bin
search-binaries-default
/usr/local/bin
/usr/bin

I tested by renaming my two parted versions to “parted” (system version in /usr/bin and patched version in /usr/local/sbin). This is how it was right after I initially compiled and installed my patched parted.

Now when I do…
$ parted -v
I get parted 3.6 back and not parted 3.6.6-c5bd-dirty.

If search-binaries-default does indeed determine the default binary search order then this makes sense. But I’m confused, I started this thread because, right after I initially compiled and installed my patched parted and had not renamed it yet, $ parted -v was seeing my patched parted v3.6.6 (in /usr/local/sbin) and not my system parted v3.6 (in /usr/bin)? With them both now named ‘parted’ again and in their same locations as always why is $ parted -v NOT now seeing v3.6.6?

Anyway, at this stage I have libparted libraries in /usr/lib and /usr/local/lib. According to this thread binaries have a linked-in idea of where to look for their libraries so I guess I should not need to worry about my system parted using libraries from the patched parted.

The accepted answer in this thread also gives a helpful explanation about having two versions of the same command installed on your system.

Flex

[Nachlese]

@flexmcmurphy

With them both now named ‘parted’ again and in their same locations as always why is $ parted -v NOT now seeing v3.6.6?

Because /usr/local/bin is searched before /usr/bin - and the first match is what you get when you just use the name.
When you use the full path, you can choose which one to use.
(requires that you even know that you now have two and can choose between them)

Search order is defined in /etc/login.defs
and
/etc/profile
/etc/profile.d/*

echo $PATH gives the current one

[Molski]

@flexmcmurphy

systemd-path | grep search-bin | sed 's/:/\n  /g'

Using this I can see on my system that (?strangely) Manjaro will look to /usr/local/bin first and then /usr/bin after that. I suppose that’s what “search-binaries-default” is about?

I believe you can ignore systemd-path since you are just running the command with your user’s shell. After all the things systemd-path can do, the end result is still the PATH environment variable for your shell. So it is just going down the list in: echo $PATH, starting with the first directory.

From what I can tell, Manjaro adds this, on login, in: /etc/profile:

    14  # Append our default paths
    15  append_path '/usr/local/sbin'
    16  append_path '/usr/local/bin'
    17  append_path '/usr/bin'

With /usr/local/sbin getting appended first.

I try to leave these files alone, and it makes dealing with pacnews easier. So instead of modifying that file, I would place PATH=/usr/local/sbin:$PATH in ~/.bashrc or whatever shell you are using to only change it for your user.

[papajoke]

@Molski

Manjaro adds this, on login, in: /etc/profile: With /usr/local/sbin getting appended first.

“flex” use KDE, and with kde, original order is set in sddm.conf (nice, today it’s the same list as profile)

[Users]
DefaultPath=/usr/local/sbin:/usr/local/bin:/usr/bin

[xabbu]

@flexmcmurphy

sudo make

Never compile software as root. Never.

@flexmcmurphy

cd /usr/local/src

Why not pick a folder in your Home?

For tests or if you the only user on the system to use this program, don’t install it system-wide. Just pick a folder in your Home. You can add it to your PATH, but this is only for your convenience and not necessary and will override the distro version unless you manually change the name.

Personally, I never install it, if it is just for a test. The make install command just copies files and creates folders. And this is the reason why this command needs sudo if you want to install a program system-wide for every user on the system. But if you the only user and you want to keep using the distro version, why install it in the first place.

The binary can be startet for the compile folder, it doesn’t need to be installed. The location of the binary differs between projects, for parted, the binary is in the parted folder within the folder the git clone command created (from your expmple in your first post /usr/local/src/parted/parted/parted).

[Aragorn]

@flexmcmurphy

How do I tell Manjaro to use the original version?

As others have tried to explain already — but in such a way that it can easily be mistaken — the rationale is as follows…

• Unlike in Microsoft Windows, UNIX systems only allow calling an executable file by its name as a command that is to be executed if this file is listed in one of the directories of your $PATH, or otherwise put, the current working directory is not part of the search path for executable files. This $PATH is a list of directories, with the listed and distinct search paths separated by a colon (“:”).

• When you issue a command at the shell prompt, the shell will first look for whether the command is internal to the shell itself — even if an executable by the same name exists in a directory of the $PATH — and if this is not the case, then it will traverse the list of directories in the $PATH, whereby the first found instance of an executable file with the name of the command is used, provided that one has the permission to execute it.

• It is always possible to invoke an executable file residing in a directory that’s not included in the $PATH, by providing an absolute or relative path to where the file is located. An absolute path is the path to the executable file from the top of the directory hierarchy (e.g. /usr/local/sbin/parted), whereas a relative path is the path to this executable file relative to the current working directory (e.g. ./scripts/somescript.sh).

• There is no need to give your locally compiled parted a different name. Either invoke it via its absolute path, or create an alias that invokes it via its absolute path, but then it is best to give the alias itself a different name, so as to not have its existence prevent you from using the system-default version of the executable.

• UNIX is a multi-user architecture, and so it is possible to set a different value for the $PATH in your own account versus the system-wide default $PATH, commonly via your ~/.bash_profile or the equivalent zsh environment file — possibly ~/.zprofile, but I use bash, so I don’t have any real experience with zsh. Command aliases and functions should go into your ~/.bashrc or ~/.zshrc, or in a third file that you source from within ~/.bashrc or ~/.zshrc.

---

@papajoke

and, you can use /opt/xxx/ for your own applications

Actually, no, that would be a bad idea.

• /opt is for applications or tools which are installed from their vendors in a binary form, but outside of the scope of the package manager — e.g. if you install a software package in binary form straight from the vendor’s website, usually by way of a self-extracting archive or a script.

• /usr/local is for software that was compiled from sources on the local machine, and/or for locally shared read-only data.

However, with regard to the latter, as @xabbu said…

@xabbu

Never compile software as root. Never.

Exactly. Unpack the source code in your home directory somewhere, compile it under your own account, and then install it, either with sudo or as root.

[flexmcmurphy]

Thank you everyone for your helpful insights.

For any other noobs like me reading this thread to clarify the steps from my first post.

Going forward I will compile code (e.g parted) into a dedicated ‘apps’ directory in my home dir, and not as root, like this…

cd /home/flex/apps/src
git clone https://git.savannah.gnu.org/git/parted.git
cd parted/
./bootstrap
./configure --prefix=/home/flex/apps --disable-static --disable-device-mapper
make

In my case I would aim to run the binary for testing purposes so would probably just run the binaries from where ever they get created.

Alternatively, to fully install I would still choose to have them install in to my home directory as follows which achieves that goal because of ./configure --prefix=/home/flex/apps above.

make install

I guess the binary will get “installed” into something like…
/home/flex/apps/[bin or sbin]/[name_of_app_binary]

Now since my aim here is for testing only and I want to ensure no overlap of an app I am testing that is also a system app I would do these steps, even though as explained above they are not strictly necessary.

1. Make sure the installed dir is NOT in my PATH. ($echo $PATH)
2. Rename the locally compiled application (e.g: parted → parted-v3.66)
3. Create an alias in my ~/.bashrc to the re-named app so I can access it from anywhere and not have to specify the full path to it.
   alias parted-v3.66="/home/flex/apps/sbin/parted-v3.66"

And Finally
To satisfy my OCD: just to go back to a question I asked above that probably came across confusing.
Firstly, I understand that /usr/sbin is symlinked to /usr/bin.

This is the first part of my $PATH:
/home/flex/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin

With these…

/usr/local/sbin/parted (v3.66)
/usr/sbin/parted (v3.6)

If I do…
$parted -v it sees (v3.66) which makes sense.

Then if I do this…

sudo mv /usr/local/sbin/parted /usr/local/sbin/parted-patched
$parted -v

What happens? I would expect it to see v3.6 now because the only parted that exists on my system is in /usr/sbin which is v3.6.

Instead within the same bash shell Konsole window/tab I get this:
bash: /usr/local/sbin/parted: No such file or directory

If I open a new bash shell window in Konsole it then works as expected and $parted -v (sees v3.6)

But again if from this new shell window if I then do this:

sudo mv /usr/local/sbin/parted-patched /usr/local/sbin/parted
parted -v

I would expect it to see v3.66 since /usr/local/sbin/parted is v3.66 and it exists now again in the file system. Instead parted -v still sees v3.6.

I guess I stumbled upon behavior of Konsole or maybe bash or something where it caches into memory the state of the file system when you initially open a command line screen? You can imagine how confusing this past few days were for me trying to process a lot of new concepts and stuff that seemed to contradict everything!

Thank you,

Flex

[Nachlese]

@flexmcmurphy

./configure --prefix=/home/flex/apps

the declaration of the --prefix=/xyz in the configure step
configures the build and determines what will happen later, when the thing is built and then installed with
make install.

That is the only step where you might need sudo (admin rights) - when it is to be installed into a directory your normal user has got no writing rights.

That is where it will be installed.
Not where you compile it.

You can build it anywhere. Doesn’t matter.
--prefix=/xyz will later install it into /xyz

./configure --help
usually tells you all about it and much more

[Molski]

The command: which <COMMAND> will tell you the full path of any executable, before you attempt to run it. Even though you should be essentially getting the same results with your test. This way works a little better; for example, if you copied/moved/linked the wrong binary. Plus, it can be used for many other purposes.

@flexmcmurphy

If I open a new bash shell window in Konsole it then works as expected and $parted -v (sees v3.6)

I don’t use KDE, but in all my experience with PATHs, modifying it is instant. Like you have echo $PATH as you want, then run the command. You still have to reload your shell?

[Nachlese]

only
which -a COMMAND
will tell you about more than the first match in your $PATH if more than one with the same name exist

[flexmcmurphy]

@Nachlese Thanks for that command, really useful to know.

@Molski It’s frustrating but yeah as described in my last post there are scenarios where I need to reload my shell. I shouldn’t have to do that, it’s like the shell is unable to keep track of changes to the file system in real time such as mv’ing a binary.

Flex

[Molski]

When you make changes in the profile or shell startup files, yes you need to reload your shell. Like many applications, you have to restart after configuration file changes.

But you can test what to set it as just by typing PATH= /usr/local/... first.