Network Routing

Table of Contents:

  1. Introduction to Routing and Its Components
  2. Tables and Chains in iptables
  3. Network Address Translation (NAT) and its Types
  4. Deep Dive into Mangle Table
  5. The ip route Command and its Use Cases
  6. Policy-Based Routing (PBR)
  7. Handling Unavailable Routes and Failover
  8. Overlapping Networks and the Role of netmap

Introduction to Routing and Its Components:

Routing is the process by which data packets are forwarded from one network to another. Key components involved:

  • Routing Table: Contains information about which path to use to reach a specific destination.
  • Metric: Indicates the “cost” of using a route.
  • Priority: Used to rank routes.

Tables and Chains in iptables

Tables

  1. Filter: The default table used for packet filtering.
  2. NAT (Network Address Translation): Used for translating network addresses.
  3. Mangle: Used for special packet alterations.
    • The Mangle table is your toolbox for specialized packet alterations, mangle has all of the chains listed above.

Chains

  • PREROUTING: First point of contact for incoming packets.
  • INPUT: For packets destined to the local machine.
  • FORWARD: For packets routed through the local machine.
  • OUTPUT: For packets originating from the local machine.
  • POSTROUTING: Last stop for outgoing packets.

Mangle PoC - TTL and phone plans when trying to tether

Every routed IPv4 hop decrements TTL. Some tethering setups historically inferred an extra downstream hop from the observed TTL; the carrier does not “boost” the packet TTL. This rule records the workaround I tested, although provider terms and network behavior vary.

How can you navigate this? By altering the TTL using the Mangle table on your computer:

iptables -t mangle -A POSTROUTING -j TTL --ttl-set 64

Mangle PoC - Routing Different Packets to Different Routes

Suppose you’re running a server with multiple internet connections, and you want to route specific packets through a different connection based on certain criteria.

First, you mark the packets:

iptables -t mangle -A PREROUTING -p tcp --dport 80 -j MARK --set-mark 1
  • In the context of iptables and network routing, marking packets means assigning them a specific identifier. This identifier can then be used by other tools or by the kernel’s routing decision mechanisms to decide how to handle the packet, so they arent permanently added to the packet once they are sent out. Marking can be set both in decimal 1 and hexadecimal 0x1 as it is merely in the representation of the same value.

You can mark packets based on a multitude of criteria. Here are some:

# Protocol Type
iptables -t mangle -A PREROUTING -p tcp -j MARK --set-mark 1
# Destination Port
iptables -t mangle -A PREROUTING -p tcp --dport 22 -j MARK --set-mark 1
# Source or Destination IP
iptables -t mangle -A PREROUTING -s 10.0.0.1 -j MARK --set-mark 1
# Interface
iptables -t mangle -A PREROUTING -i eth0 -j MARK --set-mark 1
# By mac address
iptables -t mangle -A PREROUTING -m mac --mac-source XX:XX:XX:XX:XX:XX -j MARK --set-mark 0x1
Other Real-World Uses of Mangle Table
  • QoS (Quality of Service) Implementation: By marking packets, you can prioritize certain types of traffic over others.
  • Implementing Policy-Based Routing: Directing traffic based on policies rather than simple destination-based routes.

Routes and Failover - Handling Unavailable Routes:

  1. Route Discovery: Routes are discovered in two primary ways:

    • Static Routes: Manually defined by administrators.
    • Dynamic Routes: Discovered and managed using routing protocols like OSPF, BGP, RIP, etc.

  2. Route Validity:

    • Static Routes: A static route is normally removed automatically when its local interface/link goes down. It does not, by itself, probe whether an upstream gateway or the internet beyond it still works; that needs a network manager, routing protocol, or health-check mechanism.
    • Dynamic Routes: Routing protocols have mechanisms to determine if a route (or next hop) is still valid. For instance, routing protocols often use “hello” packets to check the availability of neighboring routers. If a neighbor stops responding, the route through that neighbor is deemed invalid, and the routing protocol will adjust the routing table accordingly.

  3. Failover:

    • When the kernel removes a primary route because its interface goes down, a higher-metric route can become active.
    • A reachable Ethernet link with a dead ISP beyond it usually leaves the static route installed, so metrics alone do not provide end-to-end failover.
    • In dynamic routing setups, routing protocols can adapt quickly. They will detect failures and adjust metrics and routes in real-time to ensure continuous data flow.

The ip route command

The ip route command is utilized to manage kernel routing tables. These tables determine how packets move within and outside a network. While rules (managed by ip rule) define which table to use, the routes within those tables dictate the path of the packet.

Key Terms:

  • Metric: A value that indicates the “cost” of using a route. The lower the metric, the more preferred the route is. It’s an essential concept, especially when there are multiple possible routes to a destination.
  • Route metric: Ranks otherwise comparable routes inside one routing table; lower is preferred.
  • Rule priority: Orders Routing Policy Database (ip rule) lookups; lower numeric priority runs first. A rule selects a table, while a route in that table selects a path. Linux RPDB manual

1. Setting Up Basic Routing:

Let’s say you have two interfaces: eth0 for internal traffic and eth1 for external traffic. You can set routes as:

ip route add default via 192.168.1.1 dev eth1
ip route add 10.0.0.0/24 via 10.0.0.1 dev eth0

2. Working with Metrics and a common use case:

Consider an environment with two internet connections.

  1. Primary route through ISP A with a metric of 100.
  2. Backup route through ISP B with a metric of 200. One is the primary link, and the other serves as a backup. You can set the primary route with a lower metric:
ip route add default via PRIMARY_GATEWAY dev eth0 metric 100
ip route add default via BACKUP_GATEWAY dev eth1 metric 200

Under normal circumstances traffic uses ISP A. If its interface goes down and the route is removed, ISP B becomes best. I add an external health check when I need failure detection beyond the local link.

Routing Marked Packets with iproute2

If you have marked packets with iptables and you want them to follow a specific routing table, you’ll need to set up that routing using iproute2. Here’s a simple step-by-step for a marked packet:

# Create a new routing table
echo "200 custom_table" >> /etc/iproute2/rt_tables
# Add a route to the new table
ip route add default via GATEWAY_IP dev INTERFACE_NAME table custom_table
ip route add 192.168.2.0/24 via 192.168.2.1 dev eth2 table custom_table
# Create a rule to direct marked packets to the new table
ip rule add priority 100 fwmark 1 table custom_table

Using iptables to “mark” packets originating from a specific IP address. This mark is then used by the ip rule command to route the marked packets based on the custom rule.

Benefits:

  1. Granularity: You can use all the power of iptables to match packets. This includes not only source and destination IP but also protocols, port numbers, interfaces, etc. This method is more versatile if you plan to create more intricate policies in the future.
  2. Compatibility with other tools: Other Linux tools can also act upon these marks, so it’s a more flexible way if you have a complex system setup that requires interaction between multiple tools.

Drawbacks:

  1. Complexity: For simple routing based on source IP, it introduces an additional layer of complexity which might not be necessary.

  2. Performance: While the performance impact of using iptables for marking and then ip rule might be negligible in most cases, it’s still an extra step in the packet processing pathway.

Using ip rule with lookup:

This method bypasses the iptables mark mechanism and directly sets a routing rule based on the source IP.

Benefits:

  1. Simplicity: It’s straightforward and easy to understand, especially for scenarios that don’t require intricate packet matching criteria.
  2. Performance: Directly routing based on source IP might have a slight performance edge since it skips the packet marking step.

Drawbacks:

  1. Less Granularity: This approach is limited to the criteria that ip rule can match on, which is less versatile than the full power of iptables.

Scenario:

Suppose we have a server with two network interfaces. All regular traffic goes out via eth0, but we want all traffic originating from a specific application (that binds to a specific IP, say 192.168.10.50), to go out via eth1.

1. Create Custom Routing Tables:

First, define custom tables. Edit the /etc/iproute2/rt_tables file and add:

100     out_via_eth0
101     out_via_eth1

2. Define Routes in the Custom Tables:

# Populate the custom tables with routes
ip route add default via [Gateway_for_eth0] dev eth0 table out_via_eth0
ip route add default via [Gateway_for_eth1] dev eth1 table out_via_eth1

Replace [Gateway_for_eth0] and [Gateway_for_eth1] with the appropriate gateway IPs for eth0 and eth1, respectively.

3. Implement Policy-Based Routing Rules:

Define when each table should be used:

# Specific rule first; lower priority number wins.
ip rule add priority 100 from 192.168.10.50/32 lookup out_via_eth1

# Normal traffic can continue through the built-in main table.
ip rule show

4. Test:

Bind your application to 192.168.10.50 and observe its traffic going out via eth1 while other traffic uses eth0.

Troubleshooting IP Routes:

Networking issues often boil down to incorrect routes. Here are ways to troubleshoot:

  1. Displaying Routes: To view the current routing table, use:

    ip route show

  2. Using ip route get: This command simulates the route selection process for a given destination:

    ip route get 8.8.8.8

  3. Monitoring Route Changes: To observe changes to the routing table in real-time:

    ip monitor route

  4. Understanding Table-specific Routes: If you have custom tables, view their specific routes as:

    ip route show table custom_table

  5. Removing a Route: Sometimes, a problematic route needs removal:

    ip route del 10.0.0.0/24 via 10.0.0.1 dev eth0

Policy-Based Routing:

Policy-based routing (PBR) allows for more granular control over routing decisions, based on defined policies. Instead of merely relying on destination addresses to make routing decisions, PBR permits routing based on various criteria, like source address, packet size, protocol, or even application type. This enables administrators to design networks that can route traffic based on business needs or advanced requirements.

  1. Custom Traffic Management: Direct traffic along specific paths based on criteria other than the destination IP address.
  2. Bandwidth Allocation: Prioritize traffic for specific applications or departments.
  3. Enhanced Security: Send certain traffic through a dedicated firewall or Intrusion Prevention System.
  4. Load Balancing: Distribute traffic over multiple paths based on specific conditions.

The ip rule command

The ip rule command is a part of the iproute2 suite of utilities and is used to manage rules in the routing policy database (RPDB) in Linux. These rules determine which routing table to use for routing a given packet. This facilitates advanced routing, policy routing, and multiple routing tables on a system.

Here are some functionalities provided by the ip rule command:

  1. Display Rules: Just executing ip rule show or simply ip rule will display the current rules in order of preference.

  2. Adding Rules:

    • You can add rules based on packet marks ( look at the marking section discussed earlier ):
      ip rule add fwmark 1 table custom_table
    • Based on source or destination addresses:
      ip rule add from 192.168.1.0/24 table custom_table
ip rule add to 192.168.1.0/24 table custom_table

  3. Specifying Rule Preference: Rules are processed in order of preference. You can set this preference using:

    ip rule add pref 100 from 192.168.1.0/24 table custom_table

  4. Deleting Rules:

    • You can delete rules in a manner similar to how they are added. For example:
      ip rule del from 192.168.1.0/24 table custom_table

  5. Rule Types:

    • There are different rule types, including unicast (standard lookup in the routing table), broadcast (accept packets as broadcasts), throw (stop policy routing), and others. For instance:
      ip rule add blackhole from 192.168.2.0/24
      This would drop any packets from the 192.168.2.0/24 network.

  6. TOS (Type of Service) based rules or Differentiated Services Code Point (DSCP):

    • You can create rules based on the TOS field in IP headers:

      ip rule add tos 0x10 table custom_table

      1. 0x02 (Minimize Monetary Cost): This TOS value prioritizes minimizing cost over other considerations.
2. 0x04 (Maximize Reliability): Focuses on ensuring data reliability over other metrics.
3. 0x08 (Maximize Throughput): This value gives priority to maximizing throughput.
4. 0x10 (Minimize Delay): Prioritizes reducing delays, often used for real-time services like VoIP or video conferencing.

      The Differentiated Services Code Point (DSCP) values have largely replaced traditional TOS values. DSCP occupies the same space in the header and allows for more granular traffic classification.

    Some common DSCP values:

     Default (0x00): Standard or best-effort traffic.
 AF (Assured Forwarding): There are four AF classes, each with three drop probabilities (low, medium, high), resulting in values like AF11, AF12, AF13, AF21, etc.
 EF (Expedited Forwarding, 0x2E)**: Used for real-time, low-loss, and low-latency traffic, such as VoIP.
 CS (Class Selector values): Backward compatibility with IP precedence. They range from CS0 (routine) to CS7 (network control).

  7. Realms:

    • Realms are primarily used in conjunction with traffic control (tc) in Linux. They allow classification of routes into different realms. Rules can then be used to assign realms based on various criteria:
      ip rule add from 192.168.1.0/24 realm 10

  8. Specify Rule Protocol:

    • By default, the rule protocol is boot (indicating rules set at boot time). You can specify a different protocol if desired:
      ip rule add protocol static ...

  9. Match Rules with Incoming/Outgoing Interfaces:

    • You can create rules based on the interface through which packets enter or leave:
      ip rule add iif eth0 table custom_table
ip rule add oif eth1 table custom_table

Journey into NAT Tables

NAT table comes into play when network address translation is the game.

Source NAT (SNAT)

It’s employed when you need to change the source address of packets. Let’s say you have a local network and want to share a single public IP for internet access. When devices in your local network access the internet, their source address (local IP) needs to be translated to the public IP.

iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source PUBLIC_IP_ADDRESS

Destination NAT (DNAT)

Imagine you’re hosting several services in a local network and have one public IP. Using DNAT, you can redirect incoming traffic to the appropriate internal machine based on the port.

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination LOCAL_IP_ADDRESS

This way, any traffic coming to your public IP on port 80 gets forwarded to the local machine with the specified IP.

Overlapping Networks and netmap:

When two networks have overlapping IP ranges, direct communication can be problematic. If you try to route between them, packets might never leave the local network because the destination IP seems to be local. NAT (Network Address Translation) can be used to solve this by “mapping” one IP range to another non-overlapping range.

netmap in iptables allows you to define a 1:1 mapping of a subnet to another subnet. For instance, if you have an overlapping subnet of 192.168.1.0/24 on both sides, you can map one side’s subnet to, say, 10.10.10.0/24.

Scenario: WireGuard VPN with Overlapping Subnets:

Imagine you’ve set up a WireGuard VPN to connect two sites, Site A and Site B. Both sites independently chose 192.168.1.0/24 as their local subnet. Now, they need to communicate with each other over the VPN.

Problem: A device in Site A with IP 192.168.1.5 wants to talk to a device in Site B with IP 192.168.1.10. But when it tries to, it thinks 192.168.1.10 is on its local network, and the packet never gets sent over the VPN.

Solution: Use NAT with netmap to make Site B’s network appear as 10.10.10.0/24 to Site A and vice-versa.

Implementation:

  1. Site A Configuration. Site B appears as 10.10.20.0/24; Site A presents itself as 10.10.10.0/24 over the tunnel:

    ip route add 10.10.20.0/24 dev wg0
iptables -t nat -A PREROUTING -i wg0 \
  -s 10.10.20.0/24 -d 10.10.10.0/24 \
  -j NETMAP --to 192.168.1.0/24
iptables -t nat -A POSTROUTING -o wg0 \
  -s 192.168.1.0/24 -d 10.10.20.0/24 \
  -j NETMAP --to 10.10.10.0/24

  2. Site B Configuration. Site A appears as 10.10.10.0/24; Site B presents itself as 10.10.20.0/24:

    ip route add 10.10.10.0/24 dev wg0
iptables -t nat -A PREROUTING -i wg0 \
  -s 10.10.10.0/24 -d 10.10.20.0/24 \
  -j NETMAP --to 192.168.1.0/24
iptables -t nat -A POSTROUTING -o wg0 \
  -s 192.168.1.0/24 -d 10.10.10.0/24 \
  -j NETMAP --to 10.10.20.0/24

Communication Flow:

  1. A device at Site A with IP 192.168.1.5 wants to communicate with a device at Site B with IP 192.168.1.10.

  2. The device at Site A sends the packet to 10.10.20.10, the mapped address for Site B’s 192.168.1.10.

  3. Site A translates the source to 10.10.10.5. The packet travels over WireGuard, and Site B translates destination 10.10.20.10 back to its local 192.168.1.10.

  4. Replies from Site B use Site A’s virtual 10.10.10.0/24 range and conntrack applies the reverse translations.

These rules also need matching routes through WireGuard and scoped FORWARD accepts. I test both directions before making them persistent; the original example mapped the same source and destination ranges on both sides and could not provide distinct identities.

Optional nftables NAT Translation

I keep ip route and iptables central above. The ordinary port-forwarding model looks like this in nftables:

table ip nat_example {
  chain prerouting {
    type nat hook prerouting priority dstnat;
    iifname "eth0" tcp dport 8080 dnat to 192.168.1.10:80
  }
  chain postrouting {
    type nat hook postrouting priority srcnat;
    oifname "eth0" ip saddr 192.168.1.0/24 masquerade
  }
}

The hook and translation are explicit, while nftables sets can replace many repeated iptables rules. nftables NAT documentation


Buy Me a Coffee