iptables and ipsets

I’m dragging my work environment from “artisan system administration” to mass-managed servers. Part of this is rationalizing, updating, and centralizing management of packet filter rules on individual hosts. Like many environments, I have a list of “management IP addresses” with unlimited access to every host. Managing this is trivial on a BSD machine, thanks to pf.conf’s ability to include an outside file — you upload the new file of management addresses and run pfctl to read it. A PF rules file looks something like this:

ext_if="em0"
include "/etc/pf.mgmt.conf"
...
pass in on $ext_if proto icmp from any to any
#mgmt networks can talk to this host on any service
pass in on $ext_if from to any
...

The file pf.mgmt.conf looks like this:

table const { 192.0.2.0/24, 198.51.100.128/25 }

When I add new management addresses I copy pf.mgmt.conf to each machine, run pfctl -f /etc/pf.conf, and the new addresses can connect.

But surely there’s some similar function on a Linux box?

To complicate matters further, our environment includes both Ubuntu and CentOS machines. (Why? Because we don’t run operating systems, we run applications, and applications get picky about what they run on.) Each version has its own way of saving and restoring iptables rules. I want to use the same method for both operating systems. What we’ve used is a single rules file, /etc/iptables.rules, read by iptables-restore at boot. We specifically don’t want to trust a copy of the packet filter rules saved by the local machine, as problems can persist across reboots. The current iptables.rules looks something like this:

*filter
#mgmt addrs
-A INPUT -s 192.0.2.0/24 -i eth0 -j ACCEPT
-A INPUT -s 198.51.100.128/25 -i eth0 -j ACCEPT
#keep state
-A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
-A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p udp -m state --state NEW,ESTABLISHED -j ACCEPT
#local stuff here
...
#permit ICMP
-A INPUT -p icmp -j ACCEPT
-A OUTPUT -p icmp -j ACCEPT
-A INPUT -i eth0 -j DROP
COMMIT

I don’t want to change /etc/iptables.rules for each machine at this point. They all vary slightly. (One day the machines will be classified by roles, but we’re in an intermediate stage right now.) Instead, I want to have the list of management addresses in a separate file. I want to copy the new file to the server, run a command, and have the new list of management addresses be live.

ipsets seems to be the way to do this. Let’s find out.

On my crashbox, I’ll create an ipset. I’m using an ipset of type nethash, because it takes CIDR blocks rather than individual IP addresses. The ipset is called mgmt, just like the management addresses on my BSD machines.

# ipset create mgmt nethash

It returns silently. Did it create the ipset?

# ipset list
Name: mgmt
Type: hash:net
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 16760
References: 0
Members:

OK, it’s in memory. Now add some addresses.

# ipset add mgmt 192.0.2.0/24
# ipset add mgmt 198.51.100.128/25

Are those addresses really in the set? Let’s ask again.

# ipset list mgmt
Name: mgmt
...
Members:
192.0.2.0/24
198.51.100.128/25

Now, export this to a file.

# ipset save mgmt > iptables.mgmt.conf

I use the file iptables.mgmt.conf to mirror pf.mgmt.conf. That file should contain something like this:

create mgmt hash:net family inet hashsize 1024 maxelem 65536
add mgmt 192.0.2.0/24
add mgmt 198.22.63.128/25
add mgmt 198.51.100.128/25

Can I restore the ipset from the file? Destroy the set.

# ipset destroy mgmt
# ipset list

It’s gone. Now to restore it from memory.

# ipset restore < iptables.mgmt.conf # ipset list
...

All my rules are there.

Now, let’s teach iptables how to use an ipset. Rather than defining addresses, we use the -m set option.

# iptables -A INPUT -i eth0 -m set --match-set mgmt src -j ACCEPT

In the iptables.rule file, it would look like this.

*filter
#allow mgmt IPs
-A INPUT -i eth0 -m set --match-set mgmt src -j ACCEPT
...

When you have several management networks, this is certainly much shorter and easier to read.

When you update the iptables.mgmt.conf file, read it in with ipset restore. You must use the -! flag. This tells ipset to ignore that the ipset already exists, and restore the contents of the ipset from the file.

# ipset restore -! < iptables.mgmt.conf

I can now copy this file to my hosts, run a command, and the packet filter rules are updated, without touching my main rules file.

I don’t recall anyone using a symbol as a command-line flag like this before, but I actually kind of like this one. “I said DO IT, damn you!”