Skip to Content

linux router

In this post I review the configuration of a multihomed router on a computer running Arch Linux[^arch]. This is not a tutorial. If you would like to set up a similar router, please read the relevant documentation. This post is not finished yet.

The router (ifrit) balances connections from my LAN to the Internet over three wireless networks. ifrit connects to two of the networks via it’s own internal wireless and a USB wireless adapter. The remaining network is accessible as a gateway on a router operating in WISP mode (marid). marid also serves as the access point for my own LAN. The following diagram may put this configuration into context better than words can:

network topology

  graph switches {
  sw1 [ label="Switch 1\\n192.168.1.101" ];
  sw2 [ label="Switch 2\\n192.168.1.100" ];
  sw3 [ label="Switch 3\\n192.168.1.252" ];
  sw4 [ label="Another Switch\\n192.168.1.251" ];
  router [ label="Cisco\\nVPN Router\\n192.168.1.250" color="#cf7b7b"];
  ap1 [ label="Wireless\\nAccess Point\\n192.168.1.61" color="#a8d1b0"];
  router -- sw1;
  sw1 -- sw2;
  sw1 -- sw3;
  sw1 -- ap1;
  sw2 -- sw4;
}

I started this project with an Orange Pi Zero running Armbian Linux1. Adding the Squid proxy quickly hit the limits of what the Orange Pi could handle.

services

dhcp

ifrit uses the standard ISC DHCP Server2. The Kea DHCP server3 is slated by ISC as a replacement, and I plan to migrate to it eventually.

/etc/dhcp/dhcpd.conf

# send dhcpnak to misconfigured clients
authoritative;

# don't inform the dns server about new leases
ddns-update-style none;

subnet 172.17.7.0 netmask 255.255.255.0 {
  default-lease-time 600;
  max-lease-time 7200;
  option domain-name "styx.lan";
  option domain-name-servers 172.17.7.1;
  option routers router.styx.lan;
  range 172.17.7.100 172.17.7.200;
}

dns

For a small network I would normally use dnsmasq4. Unfortunately it does not yet support DNS-over-TLS, so here I use Unbound5.

/etc/unbound/unbound.conf

server:
    access-control: 172.17.7.0/24 allow

    do-ip6: no

    # drop id.server, hostname.bind, version.server, version.bind queries
    hide-identity: yes
    hide-version: yes

    # only answer queries from LAN interface
    interface: 172.17.7.1

    # don't include authority info in responses
    minimal-responses: yes

    num-threads: 4

    # prefetch cache entries before they expire
    prefetch: yes

    # allow only private-domains on private addresses
    private-address: 192.168.0.0/16
    private-address: 172.16.0.0/12
    private-address: 10.0.0.0/8
    private-domain: "styx.lan"

    # use updated root hints instead of builtin
    root-hints: root.hints

    # cache size.  unused memory is wasted memory :^)
    rrset-cache-size: 64m
    msg-cache-size: 32m

    # large buffer for incoming 53/udp reduces dropped messages
    so-rcvbuf: 1m

    # open sockets for every thread
    so-reuseport: yes

    # path to CA certs, for verifying TLS resolvers
    ssl-service-pem: /etc/ssl/certs/ca-certificates.crt

    # flush cache after 10K bad replies (possibly poisoned)
    unwanted-reply-threshold: 10000

    # local servers
    local-zone: "styx.lan." static

    local-data: "router.styx.lan.  IN A 172.17.7.1"
    local-data-ptr: "172.17.7.1  router.styx.lan"

    local-data: "ap.styx.lan.  IN A 172.17.6.2"
    local-data-ptr: "172.17.6.2  ap.styx.lan"

forward-zone:
    name: "."
    forward-ssl-upstream: yes

    # quad9 resolvers
    forward-addr: 2620:fe::[email protected]#dns.quad9.net
    forward-addr: [email protected]#dns.quad9.net
    forward-addr: 2620:fe::[email protected]#dns.quad9.net
    forward-addr: [email protected]#dns.quad9.net

    # cloudflare resolvers
    forward-addr: 2606:4700:4700::[email protected]#cloudflare-dns.com
    forward-addr: [email protected]#cloudflare-dns.com
    forward-addr: 2606:4700:4700::[email protected]#cloudflare-dns.com
    forward-addr: [email protected]#cloudflare-dns.com

forward proxy

To make the most of my limited connection, I use Squid[^squid] as a caching forward proxy.

routing, firewall, and traffic shaping

Routing and related services are accomplished with the network plumbing organic to Linux (Netfilter, iproute2, etc) configured with Shorewall6.

zones

The zones configuration is simple, mostly because the lack of any Internet-facing servers means there’s no need for a DMZ.

#ZONE   TYPE            OPTIONS
fw      firewall
loc     ipv4
net     ipv4

interfaces

The only unusual option in the interfaces configuration is routeback, used here for routing between the two subnets on eth0. Also note that the optional flag is used on wireless connections which may or may not be up when Shorewall loads.

?FORMAT 2
#ZONE   INTERFACE       OPTIONS
net     eth0            dhcp,tcpflags,logmartians,nosmurfs,routefilter,routeback
net     dong0           dhcp,tcpflags,logmartians,nosmurfs,routefilter,optional
net     wifi0           dhcp,tcpflags,logmartians,nosmurfs,routefilter,optional

hosts

Most router configurations will treat an entire interface as local. I do not quite trust marid (it’s a cheap router from a manufacturer known for security issues), so I mark only a specific subnet as local rather than eth0 entirely.

#ZONE   HOSTS                   OPTIONS
loc     eth0:172.17.7.0/24

policy

Other than leaving out a $FW all ACCEPT, this policy is fairly typical. It is important to follow the principle of least privelage, even where it seems futile.

#SOURCE DEST    POLICY  LOGLEVEL    RATE    CONNLIMIT
$FW     net     ACCEPT
loc     net     ACCEPT
net     all     DROP    info
all     all     REJECT  info

rules

#ACTION         SOURCE          DEST            PROTO   DEST    SOURCE          ORIGINAL        RATE
?SECTION NEW

# Don't allow connection pickup from the net
Invalid(DROP)   net             all             tcp

# DNS services
DNS(ACCEPT)     loc             $FW
DNS(ACCEPT)     $FW             loc

# Shell services
SSH(ACCEPT)     loc             $FW
Mosh(ACCEPT)    loc             $FW

# Accept ping from local
Ping(ACCEPT)    loc             $FW

# Send responses to clients
ACCEPT          $FW             loc             icmp

snat

#ACTION         SOURCE          DEST            PROTO   PORT    IPSEC   MARK    USER    SWITCH  ORIGDEST        PROBABILITY
MASQUERADE      172.17.7.0/24   dong0
MASQUERADE      172.17.7.0/24   eth0:!172.17.7.0/24
MASQUERADE      172.17.7.0/24   wifi0

providers

#NAME   NUMBER  MARK    DUPLICATE       INTERFACE       GATEWAY         OPTIONS         COPY
DONGLE  1       1       -               dong0           192.168.10.1    track,balance   -
TENDA   2       2       -               eth0            172.17.6.2      track,balance   -
WIFI    3       3       -               wifi0           192.168.43.1    track,balance   -

stoppedrules

I allow all traffic to and from eth0 when Shorewall is “stopped”, and drop the rest.

#ACTION     SOURCE      DEST        PROTO   DEST    SOURCE
ACCEPT      eth0        -
ACCEPT      -           eth0
DROP        -           -

hotwash

In no particular order, here are some of the lessons I learned putting this together:

  • Avoid distrobutions centered around one software package. I tried Zeroshell7 before settling back on Armbian. Though Fulvio’s distrobution shows great initiative, it is poor judgement to release it as a distro rather than just the PHP application it largely is. The fancy interface obscures the underlying workings just enough to make troubleshooting nigh impossible. It does not benefit from the regular updates that a larger distrobution like Armbian provides. Not to mention the hellish situation of managing connections by trying to dupe it’s strange network manager into configuring wpa_supplicant correctly.
  • Speaking of wpa_supplicant, iwd8 shows promise, but is unusable for lacking documentation. I could not find any documentation on how to duplicate some of the features wpa_supplicant supports, such as MAC address randomization / using different MAC addresses for scanning and connecting.

  1. “Armbian Linux.” Armbian. www.armbian.com [return]
  2. “ISC DHCP Server.” ISC. www.isc.org/downloads/dhcp [return]
  3. “Kea DHCP Server.” ISC. kea.isc.org [return]
  4. “Dnsmasq.” Simon Kelley. www.thekelleys.org.uk/dnsmasq/doc.html [return]
  5. “Unbound”. NLNet Labs. nlnetlabs.nl/projects/unbound [return]
  6. “Shorewall”. Shorewall. www.shorewall.net [return]
  7. “Zeroshell Linux.” Fulvio Ricciardi. zeroshell.org [return]
  8. “iNet Wireless Daemon.” The Linux Kernel Archives. iwd.wiki.kernel.org [return]