Help route internet from usb tether to lan - nat, routes & nftables

Support Network
1

Im trying to setup my box to route internet from end0 (192.168.1.6) to internet on usb0 (dhcp). Im running dns & dhcp via docker adguard - but assume thats not working for now because once the nftable rules are applied I cannot access their web interfaces. But for now ping with ip is okay.

With my current setup I can ping the internet from the ‘router’ via the interface usb0. But I cannot ping from the interface end0.

ping 8.8.8.8 -I usb0 ← works
ping 8.8.8.8 -I end0 ← Destination Host Unreachable

Do I need to setup any static routes? Or should nftables handle all the routing?

Ive tried several guides with various nftable rules, but none of them work:

https://wiki.nftables.org/wiki-nftables/index.php/Simple_ruleset_for_a_home_router

my network config:

usb0:
[Match]
Name=usb0

[Network]
DHCP=yes


end0:
[Match]
Name=end0

[Network]
Address=192.168.1.6/24

my nftables:

table inet filter {
        chain input {
                type filter hook input priority filter; policy accept;
        }

        chain forward {
                type filter hook forward priority filter; policy accept;
                iif "end0" oif "usb0" accept
                iif "usb0" oif "end0" accept
        }

        chain output {
                type filter hook output priority filter; policy accept;
        }
}
table ip nat {
        chain prerouting {
                type nat hook prerouting priority filter; policy accept;
        }

        chain postrouting {
                type nat hook postrouting priority srcnat; policy accept;
                oif "usb0" masquerade
        }
}

my routes:

default via 192.168.102.208 dev usb0 
default via 192.168.102.208 dev usb0 proto dhcp src 192.168.102.114 metric 1024 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown <- docker?
172.18.0.0/16 dev br-cc00a7d88795 proto kernel scope link src 172.18.0.1 <- docker?
192.168.1.0/24 dev end0 proto kernel scope link src 192.168.1.6 
192.168.102.0/24 dev usb0 proto kernel scope link src 192.168.102.114 metric 1024 
192.168.102.208 dev usb0 proto dhcp scope link src 192.168.102.114 metric 1024
2

There are a few things that look off to me…

If you flushed those tables, this might work a little better. (Not tested.)

sudo nft add table nat
sudo nft 'add chain nat prerouting { type nat hook prerouting priority -100; }'
sudo nft 'add chain nat postrouting { type nat hook postrouting priority 100; }'
sudo nft add rule ip nat postrouting oifname "usb0" masquerade

sudo nft add table ip filter
sudo nft 'add chain ip filter forward { type filter hook forward priority 0; policy accept; }'
sudo nft add rule ip filter forward iifname "usb0" oifname "end0" ct state related,established accept
sudo nft add rule ip filter forward iifname "end0" oifname "wlan0" accept
3

ok, its mostly working…I can access internet from other devices on the network.
I still cant ping from the router using ping 8.8.8.8 -I end0 ← should this be the expectged behaviour?

these are the rules

table ip nat {
        chain prerouting {
                type nat hook prerouting priority dstnat; policy accept;
        }

        chain postrouting {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "usb0" masquerade
        }
}
table ip filter {
        chain forward {
                type filter hook forward priority filter; policy accept;
                iifname "usb0" oifname "end0" ct state established,related accept
                iifname "end0" oifname "wlan0" accept
        }
}

The next problem I have is how to make these rules persistent and still have docker work? When i restart all the docker rules are back and the internet doesnt work.

4

I believe you need additional rules for ICMP protocols.

Ignoring IPv6, for all of ICMP you could:

sudo nft add rule ip filter icmp accept

But a more selective and safer rule should be:

sudo nft 'add rule ip filter icmp type { echo-request, echo-reply } limit rate 4/second accept'

You probably already know you always see your current rules with:

sudo nft list ruleset

So if you backup /etc/nftables.conf (Just in case). You can create a new config by:

sudo nft list ruleset > /etc/nftables.conf

Just add at the very top of that file the following lines:

#!/usr/sbin/nft -f
flush ruleset

I’ve used Docker, but I’m unsure where that would make this any different.

5

When using docker I get the following ruleset

table inet filter {
        chain input {
                type filter hook input priority filter; policy accept;
                icmp type { echo-reply, echo-request } limit rate 4/second burst 5 packets accept
        }

        chain forward {
                type filter hook forward priority filter; policy accept;
                iifname "usb0" oifname "end0" ct state established,related accept
                iifname "end0" oifname "usb0" accept
        }

        chain output {
                type filter hook output priority filter; policy accept;
        }
}
# Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
        chain prerouting {
                type nat hook prerouting priority dstnat; policy accept;
        }

        chain postrouting {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "usb0" masquerade
        }

        chain DOCKER {
                iifname "docker0" counter packets 0 bytes 0 return
                iifname "br-cc00a7d88795" counter packets 0 bytes 0 return
                iifname != "br-cc00a7d88795" tcp dport 9000 counter packets 0 bytes 0 dnat to 172.18.0.2:9000
        }

        chain POSTROUTING {
                type nat hook postrouting priority srcnat; policy accept;
                ip saddr 172.17.0.0/16 oifname != "docker0" counter packets 0 bytes 0 masquerade
                ip saddr 172.18.0.0/16 oifname != "br-cc00a7d88795" counter packets 0 bytes 0 masquerade
                ip saddr 172.18.0.2 ip daddr 172.18.0.2 tcp dport 9000 counter packets 0 bytes 0 masquerade
        }

        chain PREROUTING {
                type nat hook prerouting priority dstnat; policy accept;
                fib daddr type local counter packets 53 bytes 3496 jump DOCKER
        }

        chain OUTPUT {
                type nat hook output priority dstnat; policy accept;
                ip daddr != 127.0.0.0/8 fib daddr type local counter packets 0 bytes 0 jump DOCKER
        }
}
# Warning: table ip filter is managed by iptables-nft, do not touch!
table ip filter {
        chain DOCKER {
                ip daddr 172.18.0.2 iifname != "br-cc00a7d88795" oifname "br-cc00a7d88795" tcp dport 9000 counter packets 0 bytes 0 accept
        }

        chain DOCKER-ISOLATION-STAGE-1 {
                iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
                iifname "br-cc00a7d88795" oifname != "br-cc00a7d88795" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
                counter packets 214 bytes 13112 return
        }

        chain DOCKER-ISOLATION-STAGE-2 {
                oifname "docker0" counter packets 0 bytes 0 drop
                oifname "br-cc00a7d88795" counter packets 0 bytes 0 drop
                counter packets 0 bytes 0 return
        }

        chain FORWARD {
                type filter hook forward priority filter; policy drop;
                counter packets 214 bytes 13112 jump DOCKER-USER
                counter packets 214 bytes 13112 jump DOCKER-ISOLATION-STAGE-1
                oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
                oifname "docker0" counter packets 0 bytes 0 jump DOCKER
                iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
                iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
                oifname "br-cc00a7d88795" ct state related,established counter packets 0 bytes 0 accept
                oifname "br-cc00a7d88795" counter packets 0 bytes 0 jump DOCKER
                iifname "br-cc00a7d88795" oifname != "br-cc00a7d88795" counter packets 0 bytes 0 accept
                iifname "br-cc00a7d88795" oifname "br-cc00a7d88795" counter packets 0 bytes 0 accept
        }

        chain DOCKER-USER {
                counter packets 214 bytes 13112 return
        }
}

With the above i get no routing… So I run my rules and all the dockers stuff is replaced. Which allows routing, but I also want docker to work…

table inet filter {
        chain input {
                type filter hook input priority filter; policy accept;
                icmp type { echo-reply, echo-request } limit rate 4/second burst 5 packets accept
        }

        chain forward {
                type filter hook forward priority filter; policy accept;
                iifname "usb0" oifname "end0" ct state established,related accept
                iifname "end0" oifname "usb0" accept
        }

        chain output {
                type filter hook output priority filter; policy accept;
        }
}
table ip nat {
        chain prerouting {
                type nat hook prerouting priority dstnat; policy accept;
        }

        chain postrouting {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "usb0" masquerade
        }
}

Then there is also this note Packet filtering and firewalls | Docker Docs
So ive tried adding in a extra rule for the DOCKER-USER, however when rebooting its still not allowing the routing to work:

table inet filter {
        chain input {
                type filter hook input priority 0; policy accept;
                icmp type { echo-request, echo-reply } limit rate 4/second accept
        }
    
        chain forward {
                type filter hook forward priority filter; policy accept;
                iifname "usb0" oifname "end0" ct state established,related accept
                iifname "end0" oifname "usb0" accept
        }

        # added docker forward accept
        chain DOCKER-USER {
                type filter hook forward priority filter; policy accept;
                iifname "usb0" oifname "end0" accept
                iifname "end0" oifname "usb0" accept
        }
}

table ip nat {
        chain prerouting {
                type nat hook prerouting priority dstnat; policy accept;
        }

        chain postrouting {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "usb0" masquerade
        }
}
6

This is just a duplicate rule with a random chain name, and it will only ever use one. But it’s not even using your Docker network.

Docker installs by default bridge-utils, and uses iptables. But it does install rules to accommodate other types of networks: host, bridge, nat, etc.

It is possible to use both iptables and nftables, but there is a few things that can you have to do to make it all work. So I would probably put it all into nftables.

I do not have the time to fully set it up myself to test, but this may help you… After a Manjaro vanilla Docker install in a VM:

# Note: "iptables.rules" is a temporary file
$ sudo iptables-save > iptables.rules
$ iptables-restore-translate -f iptables.rules                                                                                                                                                                                                                   
# Translated by iptables-restore-translate v1.8.10 on Sun Nov 24 10:44:24 2024
add table ip nat
add chain ip nat PREROUTING { type nat hook prerouting priority -100; policy accept; }
add chain ip nat INPUT { type nat hook input priority 100; policy accept; }
add chain ip nat OUTPUT { type nat hook output priority -100; policy accept; }
add chain ip nat POSTROUTING { type nat hook postrouting priority 100; policy accept; }
add chain ip nat DOCKER
add rule ip nat PREROUTING fib daddr type local counter jump DOCKER
add rule ip nat OUTPUT ip daddr != 127.0.0.0/8 fib daddr type local counter jump DOCKER
add rule ip nat POSTROUTING oifname != "docker0" ip saddr 172.17.0.0/16 counter masquerade
add rule ip nat DOCKER iifname "docker0" counter return
add table ip filter
add chain ip filter INPUT { type filter hook input priority 0; policy accept; }
add chain ip filter FORWARD { type filter hook forward priority 0; policy drop; }
add chain ip filter OUTPUT { type filter hook output priority 0; policy accept; }
add chain ip filter DOCKER
add chain ip filter DOCKER-ISOLATION-STAGE-1
add chain ip filter DOCKER-ISOLATION-STAGE-2
add chain ip filter DOCKER-USER
add rule ip filter FORWARD counter jump DOCKER-USER
add rule ip filter FORWARD counter jump DOCKER-ISOLATION-STAGE-1
add rule ip filter FORWARD oifname "docker0" ct state related,established counter accept
add rule ip filter FORWARD oifname "docker0" counter jump DOCKER
add rule ip filter FORWARD iifname "docker0" oifname != "docker0" counter accept
add rule ip filter FORWARD iifname "docker0" oifname "docker0" counter accept
add rule ip filter DOCKER-ISOLATION-STAGE-1 iifname "docker0" oifname != "docker0" counter jump DOCKER-ISOLATION-STAGE-2
add rule ip filter DOCKER-ISOLATION-STAGE-1 counter return
add rule ip filter DOCKER-ISOLATION-STAGE-2 oifname "docker0" counter drop
add rule ip filter DOCKER-ISOLATION-STAGE-2 counter return
add rule ip filter DOCKER-USER counter return
# Completed on Sun Nov 24 10:44:24 2024

These are all the rules converted to nft format displayed in stdout. So you could add these rules to your existing rule set.

But I noticed you were trying to use the interface br-cc00a7d88795, so this probably won’t work for you without substituting and/or adding rules. Where is that interface from? I assume the br- means bridge.

This is me just installing Docker in a VM, so no WiFi stick or rules related to that, but they can all work together.

As you can see it installed docker0 as a bridged interface in mine:

$ nmcli conn                                                                                                                                                                                                                                                         
NAME                UUID                                  TYPE      DEVICE  
Wired connection 1  57bfee8f-4821-3c70-99f8-871796a89eb2  ethernet  enp11s0 
lo                  f29f84bc-9d59-41a5-82a7-fc685fa35133  loopback  lo      
docker0             eaac8cee-72dd-417f-a451-b5a4c7e279a5  bridge    docker0 
$ brctl show                                                                                                                              
bridge name     bridge id               STP enabled     interfaces
docker0         8000.0242b9c60ef4       no
7

Thanks a bunch! This is all working now :slight_smile:

1 Like
8

You must of filled in some blanks, so nice work.

9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.