Shiny-mirrors - A alternative to pacman-mirrors made in Rust Lang

Contributions Software
1

Hey fellow Manjaro users :wave:

I basically rewrote pacman-mirrors in Rust, but it’s not just rewrite for the sake of it. :crazy_face:
shiny-mirrors is more user oriented compared to pacman-mirrors.
I tried to make it more user-friendly not sure if i succeeded, haven’t got much of input so far so I am not 100% how should I shape it further.

The English grammar may be very very bad so… if you want to you can improve that, feel free to create PR.
My mother language is not English.

I added bunch of useful features as IPv filtering, measure method(total - includes DNS query, establishing a connection, transfer - only download itself), option to choose file size to measure with and much more.
It should be able to handle more possible errors, it shouldn’t bail on you as much as pacman-mirrors, therefore you having more chance at getting mirrors, for example with pacman-mirrors it’s happening often it can’t find any mirrors, that shouldn’t happen almost at all with shiny-mirrors, if there is mirror in the selected or gotten country from IP endpoint then it finds in what continent the country is and tries to get mirrors from that continent.
I kinda went the way by default to find the most quality mirrors and not the most recently updated as pacman-mirrors but if you want to do that you can add after refresh subcommand arg flag -u.

It’s pacman-mirrors with bunch of additional features, better error handling, type safety and much more thanks to it being written in Rust Lang.

It has autocompletion for Zsh, Fish and of course Bash.
It comes with systemd service, timer which you can enable to refresh mirrors weekly in the background. (if using shiny-mirrors service I recommend disabling the pamac-mirrorlist.timer,service)
When running first time it will find your branch from current mirrorlist, if it can’t find the branch in the mirrorlist it will ask the person with an interactive text menu.
There is also manual page man shiny-mirrors
and much more…

If you encounter any issue or have an idea for improvement don’t worry to create a ticket on the repo.

In the last commits(Already in all branches) I reworked the CLI part because Clap as of 3.0 version there become available new type based construction of the “CLI” and I decided it would be nice to make a post about shiny-mirrors on the forum because it feels much more robust with the type based CLI construction.

Examples:

My favorite command: sudo shiny-mirrors refresh -f medium -M transfer

shiny-mirrors status / shiny-mirrors – Show the status of mirrors you are using

sudo shiny-mirrors config --branch – Open TUI to change the branch

sudo shiny-mirrors refresh – Generate a mirror list for Manjaro Linux with the default behavior

shiny-mirrors config / shiny-mirrors config show – Show the settings it was able to parse from the settings file

On every subcommand you can use --help

WARNING: Don’t use the shiny-mirrors from AUR, that version is made for distros that are using Arch mirrors, so it will retrieve Arch mirrors and if you try to system update with that mirrors in local mirrorlist it can potentially break your system.

Repository: Arisa Snowbell / Shiny Mirrors · GitLab :crab:

9 Likes
2

Thanks for the hard work!
I tried shiny-mirrors and it feels simpler and runs faster than pacman-mirrors . But I have a little question, can I skip the TUI branch setting? Something like sudo shiny-mirrors config --branch unstable, without having to choose branch manually.

2 Likes
3

Thank you,
there is the possibility of it but I hidden it from --help as it should be used in scripts only really.
this way it’s in package thats in current repos its sudo shiny-mirrors config --branch_value Stable|Testing|Unstable and in the latest commits it changed to sudo shiny-mirrors config --branch-value Stable|Testing|Unstable

1 Like
4

I actually read the help document before and tried that command. At that time it didn’t work, but it works now. Thank you so much!

1 Like
5

Groundless accusations

That is not true

That is not true

Don’t spread FUD to promote your own work.

6

For the first: Thanks to Rust type safety shiny-mirrors is able to handle more cases, pacman-mirrors as I reported on the repo some problems with status and so on, thanks to rust it didn’t happen in my case

For the second: I am sorry for that, I got the understanding that there is issue finding new maintainer that it’s like not main priority, I will edit the post, I genuinely misunderstood

1 Like
7

What got me off was the leading senctences - pacman-mirrors is filled with bugs and unmaintained - and that is simply not true. It becomes apparent whrn you look at the repo.

It is bad manners to do self-promoting by critiziing other solutions - it is simply bad taste.

Your app is great - it serves a different purpose - to those interested.

pacman-mirrors provides the core mirror list just like pacman-mirrors on Arch.

The whole mirrorshopping thing has always been a decision - maintenance wise.

You get a far more stable system by sticking to one primary mirror which is stable and reliable. (mask pamac-mirrorlist.timer or create a custom pool with one mirror - which is my recommendation)

1 Like
8

I am interested in seeing which bugs you fixed. Could you provide a side by side comparison?

1 Like
9

I am apologize for saying pacman-mirrors is unmaintained, its the way I understood it from the issue on git, It was not my intention to degrade pacman-mirrors. I genuinely thought it was that way.
My understanding is that shiny-mirrors is more focusing on user and pacman-mirrors is core mirrorlist, mirrorlist control and for sys admins.

With the bugs, I as far I know pacman-mirrors has already some age, time to mature and when developing shiny-mirrors and comparing output, pacman-mirrors returned often weird information that were wrong, from that I got the feeling there are some bugs for very long time, I will edit the post further to try avoid people getting wrong image of pacman-mirrors. I exaggerated it.

1 Like
10

I haven’t keep record of that, the some I documented were about pacman-mirrors showing wrong update status and last sync but after that I stopped documenting because I was focusing on the development itself, it was on start of development of shiny-mirrors. As I understood it was fixed in newest pacman-mirrors.
I stopped documenting it because I thought pacman-mirrors is unmaintained, there wasn’t much of activity in the repo at the time.
When I noticed there was some activity I reported these two I kept images and data for.

1 Like
11

Hello
i have two questions;
is this going to replace Pacman-mirrors?
what is the equivalent of pacman-mirrors --interactive --default && sudo pacman -Syyu ?

1 Like
12

First: I don’t have an answer, I don’t decide these things or have any information on it and pacman-mirrors is more than enough sufficient and also I found out now that it is actually maintained so

Second: There is this option sudo shiny-mirrors refresh --interactive but its simpler, it doesn’t show the country or last sync, just the address of the mirror, I haven’t worked much on that one, I will probably improve that in future (You can use vim keys or arrows to navigate and spacebar to select and enter to continue)

2 Likes
13

Yes it is. :wink:

4 Likes
14

It sounds interesting, because Rust should help system/driver developers reduce memory errors that lead to security vulnerabilities. It is more safe than C/C++, especially at handling memory errors.

In benchmark programs, Rust is similarly fast as C/C++. :+1:

2 Likes
17

i’ve used this from manjaro repos since someone recommended. and find it atleast faster than pacman-mirrors, specially when i’m in a hurry and running into mirror issues, have no idea which is better over the other though.

would it possible for you to give us all options available, other than the basic help given

1 Like
18

There is a man page which you can open with this command “man shiny-mirrors”(Args when used with long names need two dashes, in the manual page it shows one) or
you can do shiny-mirrors --help, shiny-mirrors <subcommand> --help, eg.: shiny-mirrors refresh --help, shiny-mirrors config --help, etc…

2 Likes
19

Update Notice

In version that’s been already released in unstable branch atm (r246) is using different http client, as I noticed overtime the one I was using had issues with servers that are not compliant in some way to the http standard and ignored timeouts and such similar problems, therefore I changed it from ureq to reqwest, meaning it should be much more reliable and quite faster because the timeouts are working properly in reqwest so one problematic mirror shouldn’t delay the process that much as it did but it increases the size of the executable but that shouldn’t be problem in today’s age.
It’s again using rustls(instead of native one’s or openssl) with certs retrieved from the system, same configuration as previous http client.
If you encounter any issues report them on the gitlab

1 Like
20

2 Random Interesting Things about shiny-mirrors

image
image1332Ă—428 192 KB

As you can see on the screenshot, shiny-mirrors is capable of finding invalid mirror links in the local mirror list.
It’s able to distinguish between if the mirror even exists or if the path to the repo is correct - that includes the branch too

So for example if some mirror got removed from the official list of mirrors that is provided by Manjaro, it will tell you in the status command that it doesn’t exist anymore. (afaik pacman-mirrors does this case too)

And in the second case if someone plays with the local mirror list manually and forgets about it later, with the status command shiny-mirrors will inform you if some of them are invalid because of the path - including branch. (afaik pacman-mirrors ignores the path)


image
image1347Ă—264 156 KB

Another thing, if it finds that shiny-mirrors config has different branch compared to the local mirror list it will inform the user in the status command about it and advice on what action to take.

It just interesting quirks of shiny-mirrors that I haven’t shared and have no idea if anyone else found out about it so I thought I would write it out here.

2 Likes
21

Another Interesting Quirks that people probably don’t know about

shiny-mirrors finds the average mirror time of all the ranked ones and takes upper half and writes it into local mirror list and bottom half of the average is get rid of. I found this solution to be much better than setting hard limits on how many mirrors could be written into local mirror list or maximum time by which the mirrors that don’t fit are removed. As both of these options are hard limits they wouldn’t work well for everyone and could introduce problems, as such that no mirrors are written into local mirror list because some have way slower internet or with hard limit on how many mirrors it could write some very slow mirrors could get into the local mirror list causing later on problems.
Of course there is also margin error, so for Total(DNS+Download. etc…) is margin error 200ms and Transfer(Only measures download) has margin error of 75ms.

shiny-mirrors has two arguments that can change the behavior in some limited capacity,
so called

From --help argument for refresh command:

-m, --max Maximum of mirrors to write to local mirror list

and

-L, --limit Rank only N most recently synced mirrors

With --limit argument you adjust how many mirrors it should rank, and the mirrors are chosen by how recently they were synced which is provided by Manjaro api.

With --max you limit how many will be written to the local mirror list, so for example if shiny-mirrors finds 100 suitable mirrors(Doesn’t mean ranked mirrors, just one’s that fit into the all conditions) to write and you set --max to 48, it will write 48 but if there is less suitable mirrors than that it will write all of them into local mirror list.

shiny-mirrors actually doesn’t use write to disk function for ranking mirrors, everything is downloaded into a buffer in ram, and it tries the best not to touch disk at all. Meaning, no temporary files created where it downloads the testing file and so on. (Less prone to IO errors which are hard to recover from and improves ram usage by very little amount and less IO throughput in overall system)
Of course, temporary files in /tmp are written to the RAM too, but it’s useless IO on top of it with possibility of IO errors.
But ultimately it doesn’t matter in such applications as it’s almost nothing but it’s been done like so because when I already used Rust Lang(To do best practices as it’s often used for very heavy workloads so optimizing in such way is something I wanted to do to learn it basically) I thought I should limit the read and write on disk and overall IO throughput but how I said it doesn’t matter because how small the files are. So just an interesting quirk that doesn’t matter at all.

shiny-mirrors is very opinionated as it was primary meant for my use only but when I was writing it I thought I should share it when I am at it.

1 Like