[URGENT] - IPv6 mirrors not syncing anymore from repo.manjaro.org

moson · 4 March 2021 06:52

Hi,

With the introduction of DNS AAAA (IPv6) records, a lot of mirror servers are trying to sync from repo.manjaro.org via IPv6 protocol.

Now those IPv6 addresses of a lot of mirrors are not whitelisted and hence the servers are unable to connect.

@ERROR: access denied to repos from manjaro.moson.eu (2a03:4000:47:4db:d492:aaff:fed2:7b39)
rsync error: error starting client-server protocol (code 5) at main.c(1675) [Receiver=3.1.3]

@philm

btw. For the moment I’ve added --ipv4 to the rsync parameters on my mirrors to force IPv4 protocol which can still sync…

GaVenga · 4 March 2021 12:24

btw.: for DAUs= disable ipv6 in network-connections

moson · 4 March 2021 12:45

This about mirror syncing. Nothing the user can do here.

I do want my mirrors to be accessible via IPv6. Disabling it is not an option and doesn’t make sense.

GaVenga · 4 March 2021 13:36

makes sense for the user - but temporary würckaraunt bloss - nothing more, old swedish.

freggel.doe · 4 March 2021 16:13

Unexpected side effect from trying to fix the network connectivity check under some conditions:

moson · 4 March 2021 16:27

Not at all.
Again, a lot of mirrors did not sync from the main repository server anymore because with the AAAA record those servers tried to connect via IPv6 and their IPv6 addresses are not whitelisted and got blocked on the repo.manjaro.org firewall.

You as a user cannot fix by disabling IPv6 stack on your own computer. It does not matter if you talk v6 or v4 to the mirror. The mirrors are not up-date because they can’t sync!

freggel.doe · 5 March 2021 08:26

Situation still doesn’t look like there’s been much improvement. I’m surprised this hasn’t got more attention @Manjaro-Team

linux-aarhus · 5 March 2021 11:34

Even though I have been around computers for almost three (3) decades - I have completely given up on IPv6 - my old brain cannot grasp the logic - and I am sure there is some buried somewhere.

Why is it that my uex.dk mirror is not affected by this? My rsync service lives happily with the IPv4 address.

It must be related to the it-departments of the majority of the mirror providers are not aware of the issue or more like not aware that they have a huge problem with their IPv6 assignment - if they have any at all - maybe I am just the lucky guy where my ISP does not implement IPv6 for their customers - but I really don’t know. By the way I have not created any AAAA record - I cannot see what difference it makes - I get it - a lot of infrastructure with more and more IOT devices - like cameras etc - but I do fail to see why IPv6 should be necessary for every small network on the globe.

But those IOT devices should really run on private networks - not public - and as such you can build some really huge networks using the 10.x.x.x network address space but for any normal household a 192.168.x.y/24 will suffice.

I am recommending that @philm takes down the IPv6 addresses as it solves very little and is creating sync issues for almost the entire mirror infrastructure.

GaVenga · 5 March 2021 11:39

Habe ich alles gemerkt, bin ja schliesslich kein Paladin von Karl dem Großen.
Lustigerweise kriege ich zur Zeit Updates von den moson.eu und moson.org Servern,
WENN ich ipv6 deaktiviere auf meinem Computer - sonst gar keine…
.
Funnily enough, I am currently getting updates from the moson.eu and moson.de servers,
IF I disable ipv6 on my computer - else nothing …
.
(May be corona-infected - this is at the moment the way to explain everything???)

philm · 5 March 2021 12:23

I deleted the AAAA record for repo.manjaro.org on our end. Depends when the DNS providers will sync it. We currently only whitelist IPv4 addresses, however rsyncd listens to both, hence the error when a server tried to do IPv6 communication with us. Have to look into it more before enabling IPv6 for rsyncd.

[phil@development grub.d]$ nslookup repo.manjaro.org
Server:		192.168.XX.X
Address:	192.168.XX.X#YY

Non-authoritative answer:
Name:	repo.manjaro.org
Address: 116.203.249.219
Name:	repo.manjaro.org
Address: 2a01:4f8:c17:5333::1
``

stephane · 5 March 2021 12:25

there is 3 ways to works

ipv4 mode
ipv4 & ipv6 ( dual with bridge )
ipv6 ( you cant reach by bridge ipv4 )

moson · 5 March 2021 12:30

Well, yeah, because of this:

Then there must be something wrong with ipv6 in your network.
The mirror(s) definitely support it:

curl --ipv6 https://manjaro.moson.eu/pool/overlay/autogit-1.0-1-any.pkg.tar.zst --output testfile
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12540  100 12540    0     0  1530k      0 --:--:-- --:--:-- --:--:-- 1530k

GaVenga · 5 March 2021 12:35

German Telekom.
    $ curl --ipv6 https://manjaro.moson.eu/pool/overlay/autogit-1.0-1-any.pkg.tar.zst --output testfile                                                                             
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
      0     0    0     0    0     0      0      0 --:--:--  0:00:36 --:--:--     0^C
    [ xx]$ curl --ipv4 https://manjaro.moson.eu/pool/overlay/autogit-1.0-1-any.pkg.tar.zst --output testfile                                                                        [130]
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 12540  100 12540    0     0   231k      0 --:--:-- --:--:-- --:--:--  231k

linux-aarhus · 5 March 2021 12:37

We need to consider - is IPv6 really needed - and what we have just learned - what are the implications of IPv6 addressing?

I can’t say I know - just read above.

I mean in retro-perspective it may be necessary but as this topic clear indicates there’s is a lot of problems - with the international routing.

The routing issue and the IP addressing is a problem and I think that until we have a plan or we know how to solve these IP issues we should stay away from IPv6.

philm · 5 March 2021 12:44

I also specified now address = 116.203.249.219 in /etc/rsync.d.conf on the server. This should additionally prevent to listen to IPv6 requests for now. Report back if it helps or not, until all DNS providers are synced.

address

You can override the default IP address the daemon will listen on by specifying this value. This is ignored if the daemon is being run by inetd, and is superseded by the –address command-line option.

philm · 5 March 2021 12:46

@moson Can you check if your mirror has still the IPv6 address of repo.manjaro.org accessible via nslookup and see if the current setup will force your mirror automatically to communicate with our server only via IPv4?

moson · 5 March 2021 12:55

My mirrors don’t resolve the AAAA anymore. The record is already gone.
So yeah, it is now also working again without forcing --ipv4 with rsync.

We just need to wait until it propagates around the world, then things should fix itself.

GaVenga · 5 March 2021 13:08

nslookup repo.manjaro.org                                                                                                                                                [130]
Server:		192.168.xxx.x
Address:	192.168.xxx.x#yy

Non-authoritative answer:
Name:	repo.manjaro.org
Address: 116.203.249.219
Name:	repo.manjaro.org
Address: 2a01:4f8:c17:5333::1

nslookup forum.manjaro.org                                                                                                                                               
Server:		192.168.xxx.x
Address:	192.168.xxx.x#yy

Non-authoritative answer:
Name:	forum.manjaro.org
Address: 135.181.38.249
Name:	forum.manjaro.org
Address: 2a01:4f9:c010:b613::1

linux-aarhus · 5 March 2021 13:09

edit - nothing there it was my misreading

I was using a vpn and I mis-interpreted the answer - sorry.

mithrial · 5 March 2021 13:13

edit: Thought, so