Change to pacman-mirrors app

What do think?

We have a -c all and -id argument which is used to reset a custom mirrorlist. These arguments will still reset the mirrorlist to use all and every possible mirror.

We have the -f/--fasttrack argument but that argument returns mirrors which are updated on ALL branches.

Change in the default behavior

so when a user generates a mirrorlist it will use up-to-date mirrors instead of a potentially not updated mirrors. This approach will filter out poorly maintained mirrors such as belnet.be just to name an example

As is of now pacman-mirrors does not take into account if a mirror is updated for users branch. I am thinking of slight change to pacman-mirrors so when coming to the standard generation of a mirrorlist, pacman-mirrors would filter mirrors based on

  • the branch used on the system
  • the mirrors sync status

So instead of using all available mirrors which will include mirrors known to be out of sync only mirrors which has an updated syncstatus for the active branch is chosen for the mirrorlist.

Pacman-mirrors knows this due to the download of the status.json from the mirrorservice webpage.

Already done

Decided - well almost :grinning::sunglasses:

  • The --fasttrack argument will be finetuned to take into concideration the users current branch instead of all branches.

Notes in the thinking hat

  • -y/--sync becomes default.

Your Opinion

  • What is your take on the -m random - is it obsolete or what?
6 Likes

There are so many different flags and options now for pacman-mirrors, it has become very convoluted IMO, and confusing for new Manjaro users.

A detailed wiki page is required to explain the best way to use pacman-mirrors, inclusive of plenty of examples.

Personally I stopped using it and updating mirrorlist file manually when required, which is rare when using reliable mirrors.

3 Likes

I only use one mirror myself :slight_smile:

There is a lot of examples in the man page

man pacman-mirrors

I was not thinking of another flag but a change in the default behavior so when a user generates a mirrorlist it will use up-to-date mirrors instead of a potentially not updated mirrors.

This approach will filter out poorly maintained mirrors such as belnet.be just to name an example

Could be possible that less experienced users could make use of a command that actually executes a set of “hidden” commands, but in the same time someone experienced (or interested to learn) would be able to config (sort of) those comands? Like an alias for bash … Sorry for my ranting if is irrelevant …
Something like:
sudo pacman -A
that will do
sudo pacman-mirrors --get-branch -f 10 && sudo pacman -Syyu

We have the api - and there is no need to hide it.

And the latter - you are free to invent your aliases - I don’t think it’s a good idea to build into pacman-mirrors as pointed @sueridgepipe it has quite a few options already :slight_smile:

Thanks for reminding me - I quite forgot to update that.

1 Like

I agree that by default pacman-mirrors should never include out-of-date mirrors. I see so many people post on here for help, when their only problem is an out-of-date mirror at the top of their ‘/etc/pacman.d/mirrorlist’.

1 Like

If you know a mirror is significantly out-of-date I don’t know why you’d include it in any mirror list.

I agree with @sueridgepipe though, that the application has grown rather spidery and contains things that it probably shouldn’t. Why on Earth does a command-line application contain an “API mode”? What’s wrong with having one executable that does one thing (i.e. “the UNIX way”)?

It’s probably worth checking whether there are flags that duplicate functionality, and if so, deprecate them (e.g. if -c all and -d are equivalent, just keep -c all).

1 Like

Because it is used when building ISO’s

They are not

ISOs were built before there was an API mode. What does the API mode accomplish that a command-line does not? I suppose I could just look at the source…

That could be a separate wrapper (pacman-mirrors-gui) that calls pacman-mirrors in the background?

These can be covered by passing environment variables.

Something like grep branch /etc/pacman-mirrors.conf ?

Just check https://wiki.manjaro.org/pacman-mirrors only minor changes to an otherwise up-to-date wiki.

Those enhancements are fine but they possibly shouldn’t have been implemented in the first place. The discussion threads there are pretty much the definition of scope creep.

We need to make sure that any integral part of a Manjaro installation is designed and written to a high standard. That includes, for example, documenting rationale behind changes, having code reviews, ensuring testing before deployment to repos, etc. etc. This doesn’t just affect pacman-mirrors but every script and program that we include.

It’s a switch from a hobby project approach to taking it a little more seriously. Things might be slower to change, but they should work better (as well as catch bugs before packages hit ‘stable’, or even worse, installation images).


Ah, OK, so the “API mode” is just a way of “getting” and “setting” instead of using flags. Interesting design decision. I’d have thought you could accomplish the same things with either log-form flags (--get-branch) or put every flag behind that same interface, e.g. -a country=all instead of -c all

So, the current structure is kinda messy with the mix of two argument passing methods.

2 Likes

Been begging for this for a long time.

Like others have said, this had been in one capacity or another repeatedly discussed in these forums by many users, myself included. I’m glad it is now finally being discussed by the team.

There is absolutely no objective reason for fastrack to remain a secondary means of mirrorlist generation. It should be the pacman-mirrorlist advised method of mirror generation. But I don’t see why you feel a need to change anything in pacman-mirrors behavior to accomplish this. In other words, I don’t see a need to make it default behavior.

The problem with partial updates is been caused, not because pacman-mirrors lacks any feature or behavior, but because new users are been consistently misdirected by the current documentation. The Manjaro Wiki has in fact been telling users to update mirrors the wrong way:

  • The Rankmirrors page needs to be altered to replace -g with -f as the primary means of mirror updating. This is the most important change. That page is the major source of these problems.
  • The Manjaro Mirrors page needs to be changed to include the all-important information on what are out of date and updated mirrors and how to manage them.
  • The pacman overview page, must make a reference to the mirrors page. Currently its readers have no reference whatsoever to manjaro mirrors from that page, not even in the See Also section. This page is begging users to not update their mirror lists.
  • The pacman-mirrors man page needs to clarify the use of the -f flag. Currently the text is all but completely useless. The current comments in the code for the fastrack function can be used as a starting point.

The objective is to give the -f flag the importance it needs and reduce the exposition new users have to the -g, -c and -i flags that have been the source of many of their problems.

As for your proposal of always updating the mirrorlist with up-to-date mirrors, exactly what impact this would have on -c and -i usage? These two flags are very important for users in remote locations or under very poor internet access conditions that while being well aware of out-of-date mirrors still need to limit their mirrorlist, even if that means waiting a couple of days for their mirrors to update.

2 Likes

Thanks for pointing to which wiki pages needs updating.

I have updated the mentioned wiki pages.

For your thoughts on the impact - I have been there myself - carefully considering what to change.

For the --interactive argument - I have chosen not to change anything since the mirrors last_sync status is shown in the list.

For the ´-g´, ´-c´ and ´-f´ arguments the mirrorlist will be filtered according to the users branch and only up-to-date mirrors is written to the mirrorlist

Simply put - if you are on stable branch and out of 100 mirrors only 60 is up-to-date then only the 60 is considered.

The thoughts behind was that it is preferable to have up-to-date mirrors rather than a lot of them.

Also taken into consideration is the fact that mostly only the first mirror is ever used.

But that can be changed is this is not a desired behavior

So, you are really going to do this. Seems I will have to stop using pacman-mirrors. My -c mirrorlist is often composed of just two countries and I don’t want pacman-mirrors to automatically render it useless for all purposes, be it a manjaro update, or a new package installation.

I just don’t understand why you feel this must be done. You are effectively removing functionality from pacman-mirrors for the sake of hand-holding users, when in fact the problem is not how pacman-mirrors works, but how users have been instructed to do things through bad documentation (both in the wiki and the manual page). In the meanwhile a lot of users have already been doing a tremendous job on the forums teaching users about the -f flag for a good while. Users aren’t dumb. Once they are instructed on how to properly use pacman-mirrors (*), you will see a considerable drop in partial updates reports on these forums (which are already extremely low if we measure them against the manjaro population).

But hey! That’s fine. Just one question though: Shouldn’t-f also be changed to drop the code that performs the update status check?


(*) Not much different from what is going on with current kernel problem reports in Manjaro. I see a large number of clearly Linux newcomer or otherwise inexperienced users reporting problems with their non LTS kernel installation. The reason? Well I suspect that the Manjaro notifier default settings which are telling newcomers that their perfectly valid kernel LTS version needs an update, within seconds of first booting into the new system, is making users switch from the LTS version to the new kernel versions.

1 Like

It is at all not decided - I am merely experimenting - and in that process asking for input - and your input is very valuable.

And very much different - since I am actually looking for input on how to and if to do it. By the way - of course a user would not end up with an emplyt list.

And the what I released is a dev version - not a final version.

Oh, I know, I know. In retrospect I may have sounded too aggressive. That was, believe me, not at all how I was feeling when I wrote that. :slight_smile: I’ll be more careful on how I project my thoughts.

What exactly are your thought on that?

I know this isn’t directly related to this topic, but I’ve also been thinking about this for quite some time.

@linux-aarhus I don’t use pacman-mirrors very often, but I think forcing the mirror update to include only synced mirrors might not be the best option. I might want to retain the fastest and/or closest mirror even if it isn’t synced, for example. In fact, that’s what I usually do, and only use pacman-mirrors when I want to update earlier or the connection to the mirror is bad. That’s what -f is for.

Also, I’d like to pose a question: How will these changes stop the partially synced problems?

The reason for this question is sync status depends on when you build your mirror list. Every mirror syncs at a given time, and when you update you always have a change your mirror isn’t completely synced (unless you check it first). So, how will these changes stop these occurrences? Either we put the responsibility of mirror sync status check on the user or there will have to be another way of checking the sync status during an update (unless you suppose the mirror list is always built prior to updating the system).

I think he only means the if this will be the default behaviour, then -f doesn’t need to check sync status, and so that portion of code can be dropped.