Manjaro Testing ─ Parallel downloads problem

Weird issue regarding pacman 6 and parallel downloads on desktop (linux 5.12, ethernet connection, kpe plasma with networkmanager).

With ParallelDownloads set to 5, the initial sync of the databases takes around 5-10 seconds between the :: Synchronizing package databases... message appearing and the databases being synced. When the delay between messages is not present, syncing stays at 0B/s for the same 5-10 seconds:
extra 13,9 KiB 1,00 B/s --:-- [-------------------------------------] 0%
This happens with both pacman -Syu and pacman -Syyu. Mirrors have been synced several time, problem persists.

Issue is not present on laptop with wifi (linux 5.12, i3-gaps with netctl), with the databases being synced almost instantly.

Logs:
Without ParallelDownloads set: paclog.txt - Pastebin.com
With ParallelDownloads set to 5: paclog_parallel5.txt - Pastebin.com

EDIT:
Delay is not present when ParallelDownloads = 2 is set. Past 3, delay appears on desktop. Laptop never presents such delay.

I just tried to set parallel downloads to 5 (I uncommented the default parameter) and when I force sync of database it works perfectly, instead of updating each repository one by one it refreshes them all at same time.
I guess the issue is on your side. Try to restore a default pacman.conf file and go from there and see if you can reproduce the issue. Also try to force change your mirror maybe your mirror is crap and doesn’t work properly.

PS: how did you produce these logs? my pacman.log doesn’t look like this at all.

The configuration file is the same in both computers, copied from laptop when the issue emerged.

I feel i havent been clear: All mirrors are updated at the same time, I only copied just the extra repository for example.

Mirrors have been updated several timed, issue remains.

Logs produced with sudo pacman -Syyu --debug |& tee paclog_parallel.txt, as the pacman.log presented no usefull information regarding database syncing.

Issue dissapearing when ParallelDownloads = 2 is set (anything above 2 makes the issue appear again) leads me to believe it is a networking issue.

I may seem insisting, but change your mirror, force refresh its database, and see what happens. Pretty sure you’re using a crappy (because of reasons to be determined) mirror. Most likely your mirror is blocking your multiple concurrent connections or something like that but before searching for the issue on the mirror, let confirm the issue disappears on other mirrors…

Not at all, I appreciate your help!
To remove any possible mistake/misconfiguration, I compared the pacman.conf files in both machines (with diff). Files were identical.

These are the different testing methodologies i’ve used regarding mirror quality:

  • Mirrorlist file (/etc/pacman.d/mirrorlist) copied directly from laptop (where issue is not present). Database force-refreshed. Issue remained. Reducing parallel amount to 2 made delay dissapear.

  • Forced a re-sorting of mirrors (using pacman-mirrors --fasttrack, and verified afterwards sorting order was different). Database force-refreshed. Issue remained. Reducing parallel amount to 2 made issue dissapear.

This is what lead me to believe it is either a router issue (laptop is on wifi, desktop on ethernet) or a network issue (i.e., laptop runs netctl while desktop is on network-manager).
It is not possible to run ethernet/network-manager on laptop, nor is it to connect the desktop via wifi.

If you tried other mirrors and issue remains I don’t know, I can tell you the mirror I use and the works (Belgium, Futureweb server, https://mirror.futureweb.be/manjaro/testing/extra/x86_64) but that may not help in the end.

Maybe others can report if with parallel downloads set to 5 they also have issues.

For sure I see lot of 404 errors in your parallel 5 log so that may be the mirror, I don’t know (search for “error code” in the log to find all the web servers responses).

My config is KDE so I use NetworkManager too, and ethernet connection. I configured pacman-mirrors to use only HTTPS mirrors.

1 Like

I set pacman-mirrors to https-only to test if it made any difference, but nothing. Also noticed the amount of 404s in the parallel log, but as there are also a few in the sequential logs i didnt think much of it. Gonna leave paralleldownloads at 2 for now, and test it again next update. Thx again for your help :slight_smile:

Indeed, it seems to be related to connection problems.

Even here in Germany where in many places, telephone wires are misused for transferring data from and to the internet, I could not reproduce the issue.
All repositories were updated in parallel, without a visible delay.

If your computer is connected via WLAN, I’d try connecting it via Ethernet cable to the router and re-checking.

If your computer is connected via WLAN, I’d try connecting it via Ethernet cable to the router and re-checking.

As I’ve mentioned above, the problem is only present on my desktop. Laptop is the one on wifi, desktop is connected via ethernet (laptop runs netctl while desktop is on network-manager).
It is not possible to connect the desktop via wifi.

Then, it may be worthwhile to have a check at one of those speed test sites how your internet connection behaves :slight_smile:

As I said, even here in Germany, connected to the internet via telephone wire (they use to call it “DSL” :slight_smile: ), I cannot reproduce the issue.

4ms ping, 100/100 speeds. Even choosing a random german server i get 40ms ping max, with still 100/100.

When i can i will test connecting the laptop via ethernet with an adapter, i’m starting to believe it’s a router problem