The next FreeBSD book?

BSDCan hasn’t officially started, and I keep getting asked when I will write a third edition of Absolute FreeBSD?

The short answer is: I don’t know.

The slightly longer answer is: it depends in part on you.

The much longer answer is:

FreeBSD has added lots of stuff since Absolute FreeBSD came out in 2007. The big, screaming, basic change is that ZFS is really well-supported, and considered a core feature.

But you can’t install to ZFS. Or to a mirror. Or to any of the other really cool options available on FreeBSD. There’s good stuff there, but new users can’t have it.

There are ways around this. For a new user, they range from ugly to absurd. I had hopes for the new FreeBSD installer, but none of the rumored improvements have reached real users. I could write “To install FreeBSD, install PC-BSD.” But my gut rebels. If you want PC-BSD, install PC-BSD.

If I was to write a big FreeBSD book today, it would have to be “1001 ways to install FreeBSD.” It would cover getting FreeBSD onto ZFS, or mirrors, or GELI, or any of the other cool options. That’s not a fun book to write and would not be fun to read.

Some discussions at BSDCan give me hope for an improved installation process. I don’t care if it’s the current installer, or a port of PC-BSD’s installer, or a resurrected sysinstall.

I have no problem spending a chapter on planning an installation, or on things you should know before installing — just look at Absolute OpenBSD. But that chapter can’t be “Here’s FreeBSD disk management system, so you can boot off a live filesystem and manually edit disks and create zpools and GELIs and mirrors and and and…” Experienced FreeBSD users put up with this, but you can’t give this to a new user.

FreeBSD should have a decent partitioning scheme at boot. If the OpenBSD folks can manage that with their text-only installer, surely FreeBSD could do so. But at least it’s possible to partition the disk with the current installer.

So: if you’re a programmer and want a new version of the book, work on the installer.

If you’re not a programmer: bribe one.

I have some small books on FreeBSD on my schedule, but that’s a very different thing. Until the installer changes, Absolute FreeBSD 3 isn’t even on the schedule.

Books at BSDCan

I’m sure everyone who reads this blog knows I’ll be at BSDCan tonight through early Sunday morning. I will also have books for sale, however.

  • Absolute OpenBSD, $50
  • SSH Mastery, $20
  • DNSSEC Mastery (4 proof copies only), $20

    Cash only.

    If you buy a book, I’ll throw in a free Tilted Windmill Press T-shirt while supplies last. Because why not.

    I’ll also be giving away some books as review copies. If you want a review copy of Absolute OpenBSD, you’ll need to give me your email address. I’ll be handing it off to the NSP publicity department. Sorry to be a pain on that, but I have to pay for any Absolute OpenBSDs that aren’t review copies. (Or I can ship them back to the publisher, but that’s just a waste.) I’m more flexible with review copies of self-published books.

  • “Absolute OpenBSD” Kindle edition: problems and on sale

    I’ve heard from a few different readers that there are problems with the Kindle edition of Absolute OpenBSD. It’s by no means every copy in every format, but there are enough problem reports that it needs investigating.

    I’ve notified the publisher. They are investigating. When I have an update, I will post it.

    Generally, when a Kindle file has a problem, remove it from your device. When the new version appears, you can re-download it from Amazon.

    And this is the point where I mention that Absolute OpenBSD is O’Reilly’s Deal of the Day. If you want the ebook, you can get it for 50% off.

    Basic Ansible Playbooks

    Ansible is a tool for managing servers en masse, much like Puppet or CFEngine. Ansible has a shallower learning curve than either of those systems, however, and it’s idempotent. How do I know it has a shallower learning curve? Because I learned enough of it to do actual useful work in only a couple of hours.

    And before you reach for a dictionary, “idempotent” means that you can run the same script against your servers and have the end result be the same. You can run an Ansible script (or playbook) against a group of servers, take note of those that fail, modify the script, and run it again against the same group of servers, and Ansible will verify that the servers need the playbook run before running it. Only the servers that need the change will get it.

    Why would this ever happen? Maybe a datacenter is cut off by a network issue, or a LDAP server chokes, or gremlins invade a server, or a script fails because an intruder has hacked the server and this is your early warning. Your management tools need to deal with all of these.

    For example, I have an Ansible playbook that uploads a new PF configuration file and reloads the PF rules. Ansible compares the existing PF configuration to one being distributed, and if the file hasn’t changed, doesn’t reload the rules. This isn’t a huge deal for PF, but for some applications it’s vital.

    Another nice feature about Ansible is that it uses only SSH and Python. Most Unixes acquire Python as an application dependency somewhere along the way, and it’s small enough that I have no real objection to installing on servers without it. And both Puppet and CFEngine have dedicated agent software, so some kind of agent is going to wind up on the managed machine anyway.

    The biggest problem I had with Ansible was with playbooks. There’s a whole bunch of playbook documentation, and Ansible ships with sample playbooks, but they’re written somewhat like man pages, for people who already have some clue about the topic. So here are a couple really rudimentary Ansible playbooks, with explanations.

    ---
    - hosts: pf
      user: ansible
      sudo: yes
      tasks:
      - name: copy pf.mgmt.conf to servers
        action: copy src=/home/ansible/freebsd/etc/pf.mgmt.conf 
          dest=/etc/pf.mgmt.conf owner=root group=wheel mode=0644
        notify:
          - reload pf
    
      handlers:
        - name: reload pf
          action: shell /sbin/pfctl -f /etc/pf.conf

    Ansible playbooks are written in YAML, Yet Another Markup Language. I concede that my first thought on hearing the words “yes another markup language” is “that statement needs some obscenities between the second and third word.” But XML would be way overkill for Ansible. (And the YAML folks have changed their name to a different acronym, Yaml Ain’t Markup Language, trying to escape the stigma of being yet another bleeping bleepety-bleep markup language.)

    All YAML files start with a triple dash. They are space-sensitive — don’t use tabs, only spaces.

    At the top level (no indents), we have the triple dash and a dash

    ---
    - hosts: pf

    The leading hyphen basically means “new thing here,” as far as I can tell.

    At the second level of configuration, indented two spaces, we have five sections: hosts, user, sudo, tasks, and handlers.

    The hosts statement gives the name of a group of hosts. Ansible has an easily understood hosts file. This playbook applies to a group of hosts called “pf.”

    The user definition tells ansible which user to use. Ansible should SSH into the target servers as the user “ansible.”

    The sudo statement tells ansible to use sudo to perform this command. My ansible user on each host has sudo privileges, but needs a password. We’ll get an opportunity to enter the password when we run the playbook. (Ansible could log into the target server as root and avoid the need for sudo, or let the ansible user have sudo without a password, but the thought of doing either makes my spleen threaten to leap up my gullet and block my windpipe, so I don’t.)

    The tasks section is where things get interesting. We actually do stuff here. I define a third level of indentation (four spaces) after the tasks statement, and start them with a dash.

    Our first task has a name, “copy pf.mgmt.conf to servers.”

    The action that follows uses the Ansible copy module. I define a source file, a destination file, and set the owner and permissions.

    The notify statement tells the task to activate the handler named “reload pf” If the action changes a target system, the action triggers the handler. If the action doesn’t change anything, the handler is not triggered.

    We then have the handler section. It’s at the same indent level as tasks, sudo, user, and hosts, so it’s a major section. There’s one handler, “reload pf.” It performs one action, fires up a shell and runs a command.

    Taken as a whole, this playbook copies a file to all the servers in the pf group and reloads the file. The file pf.mgmt.conf contains the IP addresses of my management hosts, as I discussed elsewhere.

    Now let’s look at a slightly more complex playbook that does the same thing.

    ---
    - hosts: linux-internal
      user: ansible
      sudo: yes
      tasks:
      - name: copy iptables.mgmt.conf to servers
        action: copy src=/home/ansible/linux/etc/iptables.mgmt.conf 
          dest=/etc/iptables.mgmt.conf owner=root group=root mode=0644
        notify:
          - reload ipset mgmt
      - name: copy iptables.rules to servers
        action: copy src=/home/ansible/linux/etc/solus.iptables.rules 
          dest=/etc/iptables.rules owner=root group=root mode=0644
        notify:
          - reload iptables
    
      handlers:
        - name: reload ipset mgmt
          action: shell /usr/sbin/ipset restore -! < /etc/iptables.mgmt.conf
        - name: reload iptables
          action: shell /sbin/iptables-restore -! < /etc/iptables.rules

    This playbook updates the firewall rules on my Linux hosts. These CentOS hosts are a little simpler in that they all share a common function (virtualization). They can have a common iptables ruleset as well as a common list of management addresses. I talk about how I use ipsets, and why the rules are set up this way, elsewhere. But the important thing is:

  • This is a single procedure, so it's one playbook.
  • It updates two separate files.
  • Changing each file runs a separate command.

    So, if the iptables.rules file changes, Ansible runs iptables-restore. If iptables.mgmt.conf changes, Ansible runs ipset.

    To use these playbooks, I log in as the ansible user on the ansible server and run:

    $ ansible-playbook -K playbook-file-name.yml

    The -K tells ansible to ask for the sudo password. If your ansible user doen't need a sudo password, skip it. (But beware your spleen.)

    Ansible will log onto every host in the group, check the files, update them if needed, and run the handler commands if it updates the files.

    Ansible has many more modules than just copying files and running commands. It can assemble files from variables, install packages, and more. But a few small playbooks will get you started, and even the basic steps of managing servers firewall rules en masse will save you enough time to figure out the new modules.

    I have no doubt that Puppet and CFEngine have serious use cases and environments where they're the best choice. What my network is most short on is sysadmin brainpower, however, and Ansible is a good fit for my feeble brain.

  • iptables and ipsets

    I’m dragging my work environment from “artisan system administration” to mass-managed servers. Part of this is rationalizing, updating, and centralizing management of packet filter rules on individual hosts. Like many environments, I have a list of “management IP addresses” with unlimited access to every host. Managing this is trivial on a BSD machine, thanks to pf.conf’s ability to include an outside file — you upload the new file of management addresses and run pfctl to read it. A PF rules file looks something like this:

    ext_if="em0"
    include "/etc/pf.mgmt.conf"
    ...
    pass in on $ext_if proto icmp from any to any
    #mgmt networks can talk to this host on any service
    pass in on $ext_if from to any
    ...

    The file pf.mgmt.conf looks like this:

    table const { 192.0.2.0/24, 198.51.100.128/25 }

    When I add new management addresses I copy pf.mgmt.conf to each machine, run pfctl -f /etc/pf.conf, and the new addresses can connect.

    But surely there’s some similar function on a Linux box?

    To complicate matters further, our environment includes both Ubuntu and CentOS machines. (Why? Because we don’t run operating systems, we run applications, and applications get picky about what they run on.) Each version has its own way of saving and restoring iptables rules. I want to use the same method for both operating systems. What we’ve used is a single rules file, /etc/iptables.rules, read by iptables-restore at boot. We specifically don’t want to trust a copy of the packet filter rules saved by the local machine, as problems can persist across reboots. The current iptables.rules looks something like this:

    *filter
    #mgmt addrs
    -A INPUT -s 192.0.2.0/24 -i eth0 -j ACCEPT
    -A INPUT -s 198.51.100.128/25 -i eth0 -j ACCEPT
    #keep state
    -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
    -A OUTPUT -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
    -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT
    -A OUTPUT -p udp -m state --state NEW,ESTABLISHED -j ACCEPT
    #local stuff here
    ...
    #permit ICMP
    -A INPUT -p icmp -j ACCEPT
    -A OUTPUT -p icmp -j ACCEPT
    -A INPUT -i eth0 -j DROP
    COMMIT

    I don’t want to change /etc/iptables.rules for each machine at this point. They all vary slightly. (One day the machines will be classified by roles, but we’re in an intermediate stage right now.) Instead, I want to have the list of management addresses in a separate file. I want to copy the new file to the server, run a command, and have the new list of management addresses be live.

    ipsets seems to be the way to do this. Let’s find out.

    On my crashbox, I’ll create an ipset. I’m using an ipset of type nethash, because it takes CIDR blocks rather than individual IP addresses. The ipset is called mgmt, just like the management addresses on my BSD machines.

    # ipset create mgmt nethash

    It returns silently. Did it create the ipset?

    # ipset list
    Name: mgmt
    Type: hash:net
    Header: family inet hashsize 1024 maxelem 65536
    Size in memory: 16760
    References: 0
    Members:

    OK, it’s in memory. Now add some addresses.

    # ipset add mgmt 192.0.2.0/24
    # ipset add mgmt 198.51.100.128/25

    Are those addresses really in the set? Let’s ask again.

    # ipset list mgmt
    Name: mgmt
    ...
    Members:
    192.0.2.0/24
    198.51.100.128/25

    Now, export this to a file.

    # ipset save mgmt > iptables.mgmt.conf

    I use the file iptables.mgmt.conf to mirror pf.mgmt.conf. That file should contain something like this:

    create mgmt hash:net family inet hashsize 1024 maxelem 65536
    add mgmt 192.0.2.0/24
    add mgmt 198.22.63.128/25
    add mgmt 198.51.100.128/25

    Can I restore the ipset from the file? Destroy the set.

    # ipset destroy mgmt
    # ipset list

    It’s gone. Now to restore it from memory.

    # ipset restore < iptables.mgmt.conf # ipset list
    ...

    All my rules are there.

    Now, let’s teach iptables how to use an ipset. Rather than defining addresses, we use the -m set option.

    # iptables -A INPUT -i eth0 -m set --match-set mgmt src -j ACCEPT

    In the iptables.rule file, it would look like this.

    *filter
    #allow mgmt IPs
    -A INPUT -i eth0 -m set --match-set mgmt src -j ACCEPT
    ...

    When you have several management networks, this is certainly much shorter and easier to read.

    When you update the iptables.mgmt.conf file, read it in with ipset restore. You must use the -! flag. This tells ipset to ignore that the ipset already exists, and restore the contents of the ipset from the file.

    # ipset restore -! < iptables.mgmt.conf

    I can now copy this file to my hosts, run a command, and the packet filter rules are updated, without touching my main rules file.

    I don’t recall anyone using a symbol as a command-line flag like this before, but I actually kind of like this one. “I said DO IT, damn you!”

    Penguicon Schedule

    Someone pointed out that it would be nice to know when I’ll be doing what at Penguicon. So, here’s my schedule.

    Friday, 26 April:
    6PM-7PM opening ceremonies
    7PM-8PM Guest of Honor Social Hour
    9PM-10PM OpenBSD (kind of obligatory)

    Saturday, 27 April:
    10AM-noon: liquid nitrogen ice cream, my own flavor
    5PM-6PM: Technology Publishing in 2013

    Sunday 28 April:
    3PM-4PM: closing ceremonies

    Other than that, I’m basically free. You can find me elsewhere, as I’ll be playing free range author, but you can definitely intercept me at these times.

    At Penguicon this weekend

    I’m a guest of honor at Penguicon this weekend. I’ll be doing a one-hour talk on technology publishing in 2013. I’ll also be inventing an ice cream flavor.

    Other guests include author Jim C Hines, chef and author Jeff Potter, Arduino and Raspberry pi guru & author Maik Schmidt, hackerspace advocate and censorship activist Nick Farr, plus author and filmmaker Jason Denzel.

    Whenever I go to this type of con, I usually take in a few panels, see some cool demos, and spend a good chunk of time hanging out in the bar or lobby, wondering who all these people are. If you’re in the Detroit area, come by and say hello.

    I will also have hard copies of Absolute OpenBSD for sale at $50 each. Amazon doesn’t have them yet. No Starch doesn’t have them yet. But I have them. Cash and PayPal accepted — I had hoped to take credit cards, but the PayPal credit card app doesn’t like my knockoff Android tablet. But by showing up, you can get the book you’ve been demanding for four years now.

    And now that I’m a guest, maybe the cool kids will let me hang with them. But probably not.

    “Absolute OpenBSD” physically exists

    I hold in my hands the first copy of Absolute OpenBSD off the press.

    IMG00371The fine folks at No Starch Press did a beautiful job designing and creating the print book. I know ebooks are the coming thing, but a well-designed physical book is a delight.

    This specific copy is for Bill Allaire. I’ll have my own copy. But I’m sure Bill won’t mind if I spend a few minutes fondling his book.

    The loop is closed, the project is over. Now to lie back and let the fame and wealth just roll into — Sorry, I can’t even type that with a straight face.

    And if you ask me when the third edition is coming out, I’ll be sure to thank you. With an axe.