Dec 2011 Updates

The OpenSSH book is in copyedit. I hope to get the copyedits back this year. I’ve seen the first round of copyedits, and they don’t look too bad. Once I make the corrections, the book goes to the print-on-demand layout person and I start on the ebook conversion. The ebook should be out next month.

The best title I’ve had suggested was “SSH: You’re Doing It Wrong.” I love that title, but it’s not really appropriate. Instead, it’ll be “SSH Mastery: OpenSSH, PuTTY, Tunnels, and Keys.” That’s what the book is about, after all.

Progressing on Absolute OpenBSD 2 slowly, thanks to the holidays.

sudo auth via ssh-agent

One of the nicest things about writing a book is that your tech reviewers tell you completely new but cool stuff about your topic. While I was writing the OpenSSH book, one of the more advanced reviewers mentioned that you could use your SSH agent as an authentication source for sudo via pam_ssh_agent_auth.

I have dozens of servers. They all have a central password provider (LDAP). They’re all secured, but I can’t guarantee that a script kiddie cannot crack them. This means I can’t truly trust my trusted servers. I really want to reduce how often I send my password onto a server. But I also need to require additional authentication for superuser activities, so using NOPASSWD in sudoers isn’t a real solution. By passing the sudo authentication back to my SSH agent, I reduce the number of times I must give my password to my hopefully-but-not-100%-certain-secure servers. I can also disable password access to sudo, so that even if someone steals my password, they can’t use it. (Yes, someone could possibly hijack my SSH agent socket, but that requires a level of skill beyond most script kiddies and raises the skill required for APT.)

My sample platform is FreeBSD-9/i386, but this should work on any OS that supports PAM. OpenBSD doesn’t, but other BSDs and most Linuxes do.

pam_ssh_agent_auth is in security/pam_ssh_agent_auth in ports and pkgsrc. There are no build-time configuration knobs and no dependencies, so I used the package.

While that installs, look at your sudoers file. sudo defaults to purging your environment variables, but if you’re going to use your SSH agent with sudo, you must retain $SSH_AUTH_SOCK. I find it’s useful to retain a few other SSH environment variables, for sftp if nothing else.

Newer versions of sudo cache the fact that you’ve recently entered your password, and let you run multiple sudo commands in quick succession without entering your password. This behavior is fine in most environments if you’re actually typing your password, but as sudo will now query a piece of software for your authentication credentials, this behavior is unnecessary. (Also, this caching will drive you totally bonkers when you’re trying to verify and debug your configuration.) Disable this with the timestamp_timeout option.

To permit the SSH environment and set the timestamp timeout, add the following line to sudoers:

Defaults env_keep += "SSH_AUTH_SOCK",timestamp_timeout=0

You can add other environment variables, of course, so this won’t conflict with my earlier post on sftp versus sudo.

Now tell sudo to use the new module, via PAM. Find sudo’s PAM configuration: on FreeBSD, it’s /usr/local/etc/pam.d/sudo. Here’s my sudo PAM configuration:

auth sufficient /usr/local/lib/pam_ssh_agent_auth.so file=~/.ssh/authorized_keys
auth required pam_deny.so
account include system
session required pam_permit.so

By default, sudo uses the system authentication. I removed that. I also removed the password management entry. Instead, I first try to authenticate via pam_ssh_agent_auth.so. If that succeeds, sudo works. If not, the auth attempt fails.

Now try it. Fire up your SSH agent and load your key. SSH to the server with agent forwarding (-A), then ask sudo what you may run.

$ sudo -l
Matching Defaults entries for mwlucas on this host:
env_keep+="SSH_CLIENT SSH_CONNECTION SSH_TTY SSH_AUTH_SOCK",
timestamp_timeout=0

Runas and Command-specific defaults for mwlucas:

User mwlucas may run the following commands on this host:
(ALL) ALL
(ALL) ALL

Now get rid of your SSH agent and try again.

$ unsetenv SSH_AUTH_SOCK
$ sudo -l
Sorry, try again.
Sorry, try again.
Sorry, try again.
sudo: 3 incorrect password attempts

The interesting thing here is that while you’re asked for a password, you never get a chance to enter one. Sudo immediately rejects you three times. Your average script kiddie will have a screaming seizure of frustration.

The downside to this setup is that you cannot use passwords for sudo on the console. You must become root if you’re sitting in front of the machine. I’m sure there’s a way around this, but I’m insufficiently clever to come up with it.

Using the SSH agent for sudo authentication changes your security profile. All of the arguments against using SSH agents are still valid. But if you’ve made the choice to use an SSH agent, why not use it to the fullest? And as this is built on PAM, any program built with PAM can use the SSH agent for authentication.

Moving Static Sites from Apache to nginx

My more complex Web sites run atop WordPress on Apache and MySQL. Every so often, Apache devours all available memory and the server becomes very very slow. I must log in, kill Apache, and restart it. The more moving parts something has, the harder it is to debug. Apache, with all its modules, has a lot of moving parts.

After six months of intermittent debugging, I decided that with the new hardware I would switch Web server software, and settled on nginx. I’d like to switch to Postgres as well, but WordPress’s official release doesn’t yet support Postgres. WordPress seems to be the best of the available evils — er, Web site design tools. The new server runs FreeBSD 9/i386 running on VMWare ESXi. According to the documentations I’ve dug up, it should all Just Work.

Before making this kind of switch, check the nginx module comparison page. Look for the Apache modules you use, and see if they have an nginx equivalent. I know that nginx doesn’t use .htaccess for password protection; I must put my password protection rules directly in the nginx configuration. Also, nginx doesn’t support anything like the mod_security application firewall. I’ll have to find another way to deal with referrer spam, but at least the site will be up more consistently.

To start, I’m moving my static Web sites to the new server. (I’ll cover the WordPress parts in later posts.) I expect to get all of the functionality out of nginx that I have on Apache.

For many years, blackhelicopters.org was my main Web site. It’s now demoted to test status. Here’s the Apache 2.2 configuration for it.

<VirtualHost *:80>
    ServerAdmin webmaster@blackhelicopters.org
    DocumentRoot /usr/local/www/data/bh
    ServerName blackhelicopters.org
    ServerAlias www.blackhelicopters.org
    ErrorDocument 404 /index.html
    ErrorLog "|/usr/local/sbin/rotatelogs /var/log/bh/bh_error_log.%Y-%m-%d-%H_%M_%S 86400 -300"
    CustomLog "|/usr/local/sbin/rotatelogs /var/log/bh/bh_spam_log.%Y-%m-%d-%H_%M_%S 86400 -300" combined env=spam
    CustomLog "|/usr/local/sbin/rotatelogs /var/log/bh/bh_access_log.%Y-%m-%d-%H_%M_%S 86400 -300" combined env=!spam
Alias /awstatclasses "/usr/local/www/awstats/classes/"
Alias /awstatscss "/usr/local/www/awstats/css/"
Alias /awstatsicons "/usr/local/www/awstats/icons/"
ScriptAlias /awstats/ "/usr/local/www/awstats/cgi-bin/"
<Directory "/usr/local/www/awstats/">
    Options None
    AllowOverride AuthConfig
    Order allow,deny
    Allow from all
</Directory>
</VirtualHost>

/usr/local/etc/nginx/nginx.conf is a sparse, C-style hierarchical configuration file. It’s laid out basically like this:

general nginx settings: pid file, user, etc.
http {
    various web-server-wide settings; log formats, include files, etc.
    server {
        virtual server 1 config here
    }
    server {
        virtual server 2 config here
    }
}

The first thing I need to change is the nginx error log. I rotate my web logs daily, and retain them indefinitely, in a file named by date. In Apache, I achieve this with rotatelogs(8), a program shipped with Apache. nginx doesn’t have this functionality; I must rotate my logs with an external script.

In the http section of the configuration file, I tell nginx where to put the main server logs.

http {
...
error_log /var/log/nginx/nginx-error.log;
access_log /var/log/nginx/nginx-access.log;

Define a virtual server and include the log statements:

http {
...
    server {
        server_name blackhelicopters.org www.blackhelicopters.org;
        access_log /var/log/bh/bh-access.log;
        error_log /var/log/bh/bh-error.log;
        root      /var/www/bh/;
    }
}

That brings up the basic site and its logs. I don’t need to worry about the referral spam log, as I cannot separate it out. nginx doesn’t need ServerAlias entries; just list multiple server names.

To test the basic site, make an /etc/hosts entry on your desktop pointing the site to the new IP address, like so:

139.171.202.40 www.blackhelicopters.org

You desktop Web browser should use /etc/hosts over the DNS entry for that host, letting you call up the test site in your Web browser. Verify the site comes up, and that nginx is actually serving your content. Verify that the site’s access log contains your hits.

To rotate these logs regularly, create a script /usr/local/scripts/nginx-logrotate.sh.

#!/bin/sh

DATE=`date +%Y%m%d`

#main server
mv /var/log/nginx/nginx-error.log /var/log/nginx/nginx-error_$DATE.log
mv /var/log/nginx/nginx-access.log /var/log/nginx/nginx-access_$DATE.log

#bh.org
mv /var/log/bh/bh-error.log /var/log/bh/bh-error_$DATE.log
mv /var/log/bh/bh-access.log /var/log/bh/bh-access_$DATE.log

killall -s USR1 nginx

Run at 11:59 each night via cron(8).

59 23 * * * /usr/local/scripts/nginx-logrotate.sh

This won’t behave exactly like Apache’s logrotate. The current log file won’t have the date in its name. There will probably be some traffic between 11:59 PM and the start of the new day at 12:00AM. But it’s close enough for my purposes.

I must add entries for every site whose logs I want to rotate.

Now there’s the aliases. I don’t have awstats running on this new machine yet, but I want the Web server set up to support these aliases for later. Besides, you probably have aliases of your own you’d like to put in place. Define an alias within nginx.conf like so:

location ^~/awstatsclasses {
    alias /usr/local/www/awstats/classes/;
}
location ^~/awstatscss {
    alias /usr/local/www/awstats/css/;
}
location ^~/awstatsicons {
    alias /usr/local/www/awstats/icons/;
}

Finally, I need my home directory’s public_html available as http://www.blackhelicopters.org/~mwlucas/. This doesn’t update, but people link here. The following snippet uses nginx’s regex functionality to simulate Apache’s mod_userdir.

location ~ ^/~(.+?)(/.*)?$ {
    alias /home/$1/public_html$2;
    index  index.html index.htm;
    autoindex on;
}

For most sites, I would define a useful error page. The purpose of this site is to say “don’t look here any more, look at the new Web site,” so pointing 404s to the index page is reasonable. Defining an error page like so:

error_page 404 /index.html;

The configuration for this entire site accumulates to:

server {
    server_name blackhelicopters.org www.blackhelicopters.org;
    access_log /var/log/bh/bh-access.log;
    error_log /var/log/bh/bh-error.log;
    root      /var/www/bh/;
    error_page 404 /index.html;
    location ^~/awstatsclasses {
        alias /usr/local/www/awstats/classes/;
    }
    location ^~/awstatscss {
        alias /usr/local/www/awstats/css/;
    }
    location ^~/awstatsicons {
        alias /usr/local/www/awstats/icons/;
    }
    location ~ ^/~(.+?)(/.*)?$ {
        alias /home/$1/public_html$2;
        index  index.html index.htm;
        autoindex on;
    }
}

While I’m happy with nginx performance so far, I’m only running a couple of static sites on it. The real test will start once I use dynamic content.

Why I Give Books Away

For a year or so I’ve wanted to write a post about the impact of book reviews, specifically on Amazon book reviews, but Anne R. Allen has saved me the trouble.

In short: Amazon owns my writing career.

They make their decisions based on reviews by people like you.

And when I say “people like you,” I mean you, personally.

The biggest thing you can do to help any author is review their book in twenty words or more, and rate it four or five stars, and post it on Amazon. (Amazon considers a 3-star review not average, but negative.)

Today, for good or ill, Amazon owns the book business. Especially tech authors. We live and die by Amazon reviews. Reviews on other sites are nice too, but if the review isn’t on Amazon, it’s mostly shouting into the echo chamber.

mirroring FreeBSD-9 disks with GPT

I recently tried to mirror my hard drives in a new machine. The Handbook instructions, and those in my own Absolute FreeBSD, didn’t work well. (The Handbook now warns about this in a big, friendly, hard-to-miss red box.) So how can I mirror my disk? By using per-partition mirroring rather than full-disk mirroring.

I should note up front that this article is the result of my researches and testing. I am not a filesystem developer. I’m not even a FreeBSD committer any more. You should check the FreeBSD Handbook for updated documentation before trying this approach.

First, I need to partition my disks identically. My system has two disks, da0 and da1. da0 has an installed system, da1 is blank. Use gpart(8) to copy the GPT.

# gpart backup da0 > da0.gpt

The file should look something like this:

# cat da0.gpt
GPT 128
1 freebsd-boot 34 128
2 freebsd-ufs 162 201326464
3 freebsd-swap 201326626 8388540

Now copy this to the second disk.

# gpart restore -F /dev/da1 < da0.gpt

The -F flag tells gpart to destroy any existing GPT on the target disk. Now verify the GPT on both disks.

# gpart show
=> 34 209715133 da0 GPT (100G)
34 128 1 freebsd-boot (64k)
162 201326464 2 freebsd-ufs (96G)
201326626 8388540 3 freebsd-swap (4G)
209715166 1 - free - (512B)

=> 34 209715133 da1 GPT (100G)
34 128 1 freebsd-boot (64k)
162 201326464 2 freebsd-ufs (96G)
201326626 8388540 3 freebsd-swap (4G)
209715166 1 - free - (512B)

These look pretty identical to me. I now have a separate device node for each partition on each disk. Mirroring these works much like mirroring an entire disk.

Unlike mirroring MBR disks, to mirror GPT partitions you must be in single-user mode. Now that GPT is the default partitioning scheme I’m sure someone will figure out a clever way around this, but for now reboot into single-user mode.

# gmirror label -vb round-robin p1 /dev/da0p1
# gmirror label -vb round-robin p2 /dev/da0p2
# gmirror label -vb round-robin p3 /dev/da0p3

/dev/da0p3 is the default swap partition. You must decide if you want to mirror your swap partitions as well. I’m choosing to do so. If one of my disks fails, the system has a fighting chance to continue running. Having two independent swap areas, one on each disk, means that a disk failure will yank a swap space out from under the otherwise-working system.

Now add the second disk’s partitions to your mirror devices.

# gmirror insert p1 /dev/da1p1
# gmirror insert p2 /dev/da1p2
# gmirror insert p3 /dev/da1p3

This will mirror your boot blocks, your root disk, and your swap space.

Now you must update /etc/fstab to boot from your mirror. The tricky bit here is that the system now has /dev/da0p2 mounted as root, read-only. You don’t want to write to /dev/da0p2 again; all writes should go to the mirror. Instead, mount /dev/mirror/p2 to a temporary location.

# mount /dev/mirror/p2 /mnt
# cd /mnt/etc
# cp fstab fstab-old
# ee fstab

vi requires a read-write /var/tmp, but ee works just fine.

Let the system run until the disks are synchronized. Check your disk status with “gmirror status.”

# gmirror status
Name Status Components
mirror/p1 COMPLETE da0p1 (ACTIVE)
da1p1 (ACTIVE)
mirror/p2 DEGRADED da0p2 (ACTIVE)
da1p2 (SYNCHRONIZING, 17%)
mirror/p3 DEGRADED da0p3 (ACTIVE)
da1p3 (SYNCHRONIZING, 7%)

After your disks synchronize, reboot. Don’t just exit single-user mode, as you have the wrong root partition mounted. (I found that rebooting before mirror synchronization meant the system came up with a read-only /.)

Your drives are now mirrored. Don’t forget to add mirror checks to your daily status mails. Add the following to /etc/periodic.conf:

daily_status_gmirror_enable="YES"

Hopefully, we’ll have a faster way to do this soon.

UPDATE 7/12/2011: You must have geom_mirror_load=YES in /boot/loader.conf to run gmirror commands.

Recovering from Failing to Mirror Disks on FreeBSD 9.0-RC2

I’m installing a new FreeBSD server, and want to mirror the root disks. According to the instructions in the Handbook and my own Absolute FreeBSD, it’s a simple process. The instructions are not valid for FreeBSD 9, however. It was late. I was tired. I tried anyway.

The first clue should have been that the disk devices now have different names. Rather than /dev/da0s1, they now look like /dev/da0p1. What difference does a letter make? Well, my test instance is virtualized. I took a snapshot and tried to follow the geom_mirror instructions, including updating /boot/loader.conf and /etc/fstab. My next boot failed with:

GEOM: mirror/gm0: corrupt or invalid GPT detected.
GEOM: mirror/gm0: GPT rejected -- may not be recoverable

Cue the familiar sinking feeling in my gut.

Reading the release notes tells me that the new installer writes GPT partitions. It’s about time this change was made; GPT has been used on non-x86 hardware for years now, and overcomes many of the limitations of MBR partitions.

But geom_mirror and GPT cannot coexist on whole disks. Both write to the last sector of the disk. If you follow the mirroring instructions in the Handbook, blithely ignoring the fact that the device names have changed, you overwrite the GPT.

Oops. I’d already installed a bunch of software on this machine. I’d rather not redo that. Let’s see how to recover.

Fortunately, recovery is fairly easy. Boot the installation CD, but rather than installing, choose the live system. You’ll get a command prompt.

Now look at your disk’s GPT.

# gpart show
=> 34 209715133 da0 GPT (100G) [CORRUPT]
...

The scary bit is the last word. You can’t boot a corrupt disk. Try to recover the GPT.

# gpart recover da0
da0 recovered
#

Another gpart show should show the word CORRUPT is missing.

With the FreeBSD 9 installer, the disk’s root filesystem is /dev/da0p2.

# mount /dev/da0p2 /media
# cd /media/etc
# cp fstab-old fstab
# cd /media/boot
# vi loader.conf

Remove the geom_mirror_load=YES line from loader.conf.

Theoretically, you’ve recovered. Reboot the live CD, boot onto the hard disk. If you were following the instructions, this error should be recoverable.

I must still figure out how to mirror my boot disk correctly, but this at least got the system back up.

notes from my FreeBSD and Nagios upgrade

My Nagios system ran FreeBSD-current/i386 from October 2010 and Nagios 3.0.6. Business factors drove me to make some changes, and I decided to upgrade the server before making those changes. Here’s some things I observed. I don’t know if these is useful to you, but I’ll need them for other upgrades, so what the heck.

Back up before you start. (Yes, obvious, but everyone needs a reminder.)

Building 9-stable on a -current box that old is tricky. You have to do a variety of ugly things. So don’t. I NFS-mounted another machine running 9-BETA2/i386 and installed from that.

Remove the old libraries and obsolete programs from the core system. While you have a full backup, I find it useful to have a separate, convenient backup of removed libraries on the existing system.

# cd /usr/src
# make check-old-libs | grep '^/' | tar zcv -T - -f $HOME/2010Oct-old-libs.tgz
...
# yes | make delete-old-libs

In the event that I cannot recompile some program for FreeBSD 9, I can install the necessary libraries under /usr/lib/compat and get on with my life.

I ran portmaster-L > ports.txt to get a list of all installed software in hierarchical order, deleted what I didn’t need any longer, then used portmaster -d --no-confirm portname on my leaf ports.

I had trouble building a couple of ports. I elected to use packages for these ports. FreeBSD-9’s packages are built against Perl 5.12. In 2010, they were built against Perl 5.8. It was simpler to remove all Perl ports and reinstall them from scratch. The ports that were giving me trouble worked fine with the newer Perl.

Then there’s Nagios. Ah, there’s nothing like upgrading Nagios. Actually, the Nagios upgrade itself ran perfectly with portmaster. The problem with the upgrade is all of the additional NagiosExchange scripts I installed. Lots of them ran fine under Perl 5.8, but choked when run by Nagios in Perl 5.12. The problem scripts started with #/usr/bin/perl -w. By removing the -w (warnings) flag, they ran under Nagios again.

When you reactivate Nagios after this upgrade, either turn off email or redirect all email to /dev/null. Do not leave email on. Nagios might well generate spurious errors, spam your coworkers, and cause either alarm or annoyance, depending on their temperament.

Once I fixed all the scripts that were failing, Nagios generated intermittent errors. All of the scripts that failed were SNMP-based. I ran snmpwalks from the Nagios box, and they all died partway through. I ran tcpdump -vv -i em0 udp port 161 on the target machines, and saw that they all reported “bad UDP checksum.” The server was still running 9-BETA2. Rather than tracking down an error on an older version, I upgraded the system to 9-RC2. The problem disappeared. I dislike not understanding the problem’s cause, but obviously someone else fixed it between BETA2 and RC2.

The only plugins that still failed were check_snmp_proc and check_snmp_disk, from the nagios-snmp-plugins port. Every one of them failed consistently.

Running the plugins by hand showed that they were generating correct answers, but they were also picking up MIB file errors from my $MIBS and $MIBDIRS. I have a whole bunch of MIB files that I use for developing and testing Nagios plugins. I normally restart Nagios with sudo. On a hunch, I used su - to become root with a clean environment and restarted Nagios. The errors stopped, and Nagios ran perfectly.

I suspect that this is the same problem that broke the perl -w plugins. The newer Nagios apparently chokes on extra debugging output. I’ve gone through the release notes for the versions I skipped, but didn’t find that. In all fairness, I probably just missed it.

FreeBSD 9 PF macro & table changes

I secure my BSD servers with PF. In FreeBSD 9, PF has been updated to the same version as in OpenBSD 4.5.

I use lists in my PF configuration, as shown in this /etc/pf.conf snippet:

mgmt_hosts="{ 10.0.1.0/24, 172.19.8.0/24}"
...
pass in on $ext_if from $mgmt_hosts
...

When I have new management hosts, I add their IP address or subnets to the mgmt_hosts list. When PF reads this configuration file, every place that a rule references the list, an additional rule is created for each member of the list. Here, every subnet in the mgmt_hosts list gets a “pass in” rule. When I list these rules on a running FreeBSD 8 host, they’ll look something like this:

# pfctl -sr
...
pass in on em0 inet from 10.0.1.0/24 to any flags S/SA keep state
pass in on em0 inet from 172.19.8.0/24 to any flags S/SA keep state
...

Very useful for maintaining a readable rule file.

I updated a host to FreeBSD 9, and saw the following in my rules.

# pfctl -sr
...
pass in on vr0 inet from <__automatic_4c6aed29_0> to any flags S/SA keep state
...

Wait a minute. What is this __automatic crap? And where are my management hosts?

This version of PF automatically converts lists to tables. If you have a big rule set, using a table makes the rules shown by pfctl more readable. (I seem to recall that tables perform better than lists, but I can’t find a reference for that, so take that with a grain of sand.)

You can name tables, but tables created by PF have a name that starts with __automatic. To view all the tables, run:

# pfctl -sT
__automatic_4c6aed29_0
#

To see the hosts in this table, use pfctl’s -t and -T flags.

# pfctl -t __automatic_4c6aed29_0 -T show
10.0.1.0/24
172.19.8.0/24
#

Wow. This works, but it doesn’t look like fun. If I have to routinely type __automatic_4c6aed29_0, I will increase my subvocalized swearing by at least ten percent. But it does not interrupt service. Old rule sets continue to work. (I don’t mind needing to update my rules with a new OS version, but I need to know about it beforehand rather than just blindly updating.)

To make my life easier I can convert my PF rules to use tables instead of lists. Here’s the same pf.conf using a table instead of a list.

table <mgmt_hosts> const {10.0.1.0/24, 172.19.8.0/24}
...
pass in on $ext_if from <mgmt_hosts>
...

Unlike a list, a table is explicitly declared as a table. The name always appears in angle brackets.

I use the const keyword to tell PF that the contents of this table cannot be changed at the command line. PF tables can be adjusted at the command line without reloading the rules, which is a handy feature for, say, automatically blocking port scanners, feeding IDS data to your firewall, or DOSing yourself.

When I look at my parsed rules now, I’ll see:

# pfctl -sr
...
pass in on vr0 from <mgmt_hosts> to any flags S/SA keep state
...

I can now read my rules more easily.

(Bootnote: OpenBSD just came out with 5.0, so FreeBSD 9 is five versions behind. OpenBSD PF develops quickly. But thirty-month-old PF is better than a lot of other firewall software.)

Unlicensed Book Downloads and the Writer

(Anyone who is a big enough fan of my work to actually track down this blog is almost certainly not the target of this rant. But today, it happened one too many times.)

I had a little bit of writing time this morning before work. How did I spend it? Sending DMCA takedown notices. You can get my books for free. Even the brand new ones. They are frequently scanned and uploaded to file sharing sites, sometimes even before I get my author’s copies. I send out DMCA notices when I find them, if the host site is in the US.

What is the real impact of illicit book downloads on me as a writer?

Let’s get some of the bogus arguments out of the way.

The word piracy is ridiculous in this context. Theft is better, but that word implies scarcity. If you take a book I wrote from the store without paying for it, that’s theft. Electronic books are post-scarcity, in a certain sense. (The writing of the book is scarce, additional copies are not.) For downloading of electronic books without paying the publisher I prefer unlicensed or illicit, which aren’t perfect, but feel closer than any of the other popular alternatives.

I don’t like the DMCA, and I strongly disagree with its technological circumvention provisions. If you buy something I wrote in ebook form, I don’t care if you have a copy on every device you own or if you print it out or if you use the Kindle loan feature and get a friend to read it. If you buy something I write as a physical book, please loan it out, mark it up, photocopy key pieces and hang them above your desk, whatever. If you buy one of my physical books direct from my publisher, they’ll give you the ebook version for free, giving you the best of both worlds. But the DMCA takedown notice is the tool by which sites like scribd and tumblr accept notifications, so I use it.

So, what about my books? How does this affect me?

Writing a book is like staying on a diet. Every day, you decide you’re going to write instead of doing something else.

Writing books takes time. I have a day job. As day jobs go, it’s pretty good. I get the tools I need to do my work. I don’t have bogus meetings or daft cow-orkers. I get to choose most of the technologies I work with. Fearless Leader doesn’t call me in the middle of the night for bogus emergencies. I choose my hours. I have a private office for that couple of days a week where I condescend to grace the office with my presence. And on those days, Fearless Leader usually buys lunch. The hours are not ghastly, as in some companies, but it’s a full-time job.

When people say “Hey, did you see that show last night?” I say “No, I was writing.”

When the missus suggests I spend the evening watching movies with her, some nights I say “I really need to get some writing done.”

I just moved. My new office has floral wallpaper. I detest wallpaper. Even wallpaper without cheery climbing blue and red roses. It drives me batty. I could spend my free time for a couple of weeks and transform the room into an almost elegant techie’s office.

But moving has delayed my current books unduly. I know people are eagerly awaiting my next books. They tell me so. Repeatedly. At length. So I live with the wallpaper, and write.

I use SSH every day, but I don’t use every piece of its functionality. I’ve never needed to use a SSH VPN. To write that chapter of the OpenSSH book, I spent two weeks of “writing time” getting SSH VPNs working between Ubuntu, FreeBSD, and OpenBSD machines. I use OpenBSD daily, but I don’t use systrace. I use Apache, but OpenBSD just imported nginx. I have to figure out all of these things, and understand them well enough to explain them to you. More time.

If I just wrote fiction, I wouldn’t have to fanny about with packet sniffers and debugging logs. But fiction requires lots of research and preparation. The time is spent differently, but it’s still spent.

That’s time I could be hanging out with my family, or at the dojo, or with friends, or even watching some of the TV series I’ve never seen but that friends have raved about (Firefly, Buffy, X-files, whatever the current hit is). Instead, I’m writing.

I enjoy writing, but there’s a lot I want to write that’s much easier than technology books. And there’s a difference between writing something for myself, and writing something of sufficient quality that I can legitimately offer it to others.

The fact that my books can be fun to read doesn’t mean that they’re fun books. My books are meant to help you make money. Maybe that money is your salary, maybe it’s for your the company. Maybe the financial impact isn’t direct, but my goal is that when you finish reading one of my books, you will be more knowledgeable, more highly skilled, and a more valuable technologist. Transforming your skills into cash is your job.

Cutting out the people who help you improve yourself is downright disrespectful.

It’s been suggested that I put up a “tip jar,” so illicit downloaders can throw me a few bucks. Unfortunately, that ignores all the other people who go into making my books a success. My NSP books are professionally edited, copyedited, tech edited, and designed. I cannot in good conscience just cut them out. That would be just as disrespectful.

Losing money is unpleasant. But when someone says “I have so little respect for the year of your life that you spent working on this book that I’m going to give it away,” that’s downright insulting. Personally offensive. Disrespectful.

The greatest tool any of us have is enthusiasm for our work. Every time I find where someone has uploaded one of my books without permission, it drains my enthusiasm. Tonight, I really should finish up the tech edits on the OpenSSH book so it can go to copyedit. But I think those edits will wait. I’m going to dinner with the missus instead.

UPDATE 2015-02-10: I put up a tip jar.

Installing a DragonFly BSD Jail

I’m installing a jail on a freshly upgraded DragonFly BSD 2.13-DEVELOPMENT box. There’s instructions in the DragonFly manual, and on the Web site. They’re fine as far as they go, but to make the jail truly useful you need to do a little more.

Before starting, decide some important facts about your jail.

  • Root directory for the jail filesystem
  • IP address used by the jail
  • hostname of your jail
  • My jail hostname will be mwltest4, on the IP 192.0.2.9, in the directory /jail/mwltest4.

    A jail requires exclusive use of a single IP address. That IP must be bound to the server as an alias. Make an appropriate alias entry in /etc/rc.conf. Note that an alias needs an all-ones netmask. While we’re there, enable jails and tell the host server that we’re building the jail mwltest4.

    ifconfig_em0_alias0="inet 192.0.2.9 netmask 255.255.255.255"
    jail_enable="YES"
    jail_list="mwltest4"

    rc.conf also needs entries for each jail, so that the various jail management utilities can find and configure the jail.

    jail_mwltest4_rootdir="/jail/mwltest4"
    jail_mwltest4_hostname="mwltest4"
    jail_mwltest4_ip="192.0.2.9"

    Start by seeing what network ports your server listens on. I’ve removed all of the entries with remote addresses, because those are live network sessions; I’m only interested in what ports the server is listening on.

    # sockstat -4
    USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
    ...
    root sendmail 670 4 tcp4 127.0.0.1:25 *:*
    root sshd 656 5 tcp4 *:22 *:*

    Any entry where the local address is an asterisk followed by a colon and a port number will be a problem. We need to bind those daemons to the server’s main IP address. In this example, the only problem daemon is SSH. Bind SSH to a single IP address with a ListenAddress directive in /etc/ssh/sshd_config.

    ListenAddress 192.0.2.8

    Run /etc/rc.d/sshd restart, and sshd will bind only to the specified IP.

    I want my jails on their own filesystem, so I create a new HAMMER PFS and a directory for this particular jail.

    # hammer pfs-master /jail
    Creating PFS #9 succeeded!
    /jail
    sync-beg-tid=0x0000000000000001
    sync-end-tid=0x00000001068ea510
    shared-uuid=34cc9fbe-ffc2-11e0-9527-010c29ce51d2
    unique-uuid=34cc9fdd-ffc2-11e0-9527-010c29ce51d2
    label=""
    prune-min=00:00:00
    operating as a MASTER
    snapshots directory defaults to /var/hammer/

    # mkdir /jail/mwltest4

    Now install the userland, exactly as per the jail instructions.

    # setenv D /jail/mwltest4
    # cd /usr/src/
    # make installworld DESTDIR=$D

    Go get more caffiene. By the time you return you should see:

    ===> etc
    ===> etc/sendmail
    install -o root -g wheel -m 644 /usr/src/Makefile_upgrade.inc /jail/mwltest4/etc/upgrade/
    #

    It finished successfully. Now install /etc.

    # cd etc/
    # make distribution DESTDIR=$D -DNO_MAKEDEV_RUN

    Now mount a device filesystem for the jail.

    # cd $D
    # ln -sf dev/null kernel
    # mount_devfs $D/dev

    Edit /etc/fstab to have the host mount the jail devfs whenever the system starts.

    devfs /jail/mwltest4/dev devfs rw 0 0

    Our jail should be ready. Start it in single-user mode.

    # jail /jail/mwltest4/ mwltest4 127.0.0.1,192.0.2.9 /bin/sh
    # uname -a
    DragonFly mwltest4 2.13-DEVELOPMENT DragonFly v2.13.0.49.gf6ce8-DEVELOPMENT #0: Tue Oct 18 10:51:40 EDT 2011 mwlucas@mwltest2.blackhelicopters.org:/usr/obj/usr/src/sys/GENERIC i386
    #

    Before starting your jail in multiuser mode

  • enable SSH
  • configure /etc/resolv.conf
  • set a root password
  • and add a user
  • As I use LDAP for central account administration, but the jail isn’t yet LDAPilated, I manually set my new user ID to be identical to that on the host, and I add that account to the wheel group. Also modify /etc/ssh/sshd_config to listen only to the jail’s IP address. (While this isn’t strictly necessary, it will simplify managing the host server.)

    On the host, with my unprivileged account, I run:

    $ cp -rp .ssh /jail/mwltest4/usr/home/mwlucas/
    $ cp .cshrc /jail/mwltest4/usr/home/mwlucas/

    My jail account now has my authorized_keys file and my SSH configuration, with correct permissions, along with my preferred shell environment.

    Start the jail in multiuser mode:

    # /etc/rc.d/jail start mwltest4
    Configuring jails:.
    Starting jails: mwltest4.
    #

    I can now SSH to the jail, become root, and install pkgsrc.

    # cd /usr/src
    # make pkgsrc-create
    If problems occur you may have to rm -rf pkgsrc and try again.

    mkdir -p /usr/pkgsrc
    cd /usr/pkgsrc && git init
    git: not found
    *** Error code 127

    Stop in /usr.

    Crap. The DragonFly install installs git via package as part of the OS install. git is used for installing pkgsrc. You use pkgsrc to install git. How can we bootstrap git? pkg_radd lets you install remote packages, but it is built on pkg_add, part of pkgsrc.

    Find a FTP server (or mount an ISO) with the version of the scmgit package that runs on your host server. I would up getting the scmgit-base-1.7.4.1 package from the 2011Q1 pkgsrc. This is the same package that was originally installed on my DragonFly machine, and it still runs on the DragonFly installed on this host, so it should be okay.

    # pkg_add -f -P /jail/mwltest4/ ftp://ftp.allbsd.org/pub/DragonFly/packages/i386/DragonFly-2.10/pkgsrc-2011Q1/devel/scmgit-base-1.7.4.1.tgz
    pkg_add: Warning: package `scmgit-base-1.7.4.1' was built for a platform:
    pkg_add: DragonFly/i386 2.10.0 (pkg) vs. DragonFly/i386 2.13 (this host)
    pkg_add: Warning: package `p5-Error-0.17016nb1' was built for a platform:
    pkg_add: DragonFly/i386 2.10.0 (pkg) vs. DragonFly/i386 2.13 (this host)
    ...

    You’ll see many more warnings. The package wants to install TK and Python, but those packages are not available on this particula FTP server. But the -f flag means “Go ahead and install even if some dependencies are missing.” I use the -P to assign the package a new installation root directory in my jail’s root.

    Do I like these errors? No. But if I can install a working git, I can install pkgsrc and build a current package with all the dependencies. Log back into the jail and see if it works.

    # cd /usr
    # make pkgsrc-create
    If problems occur you may have to rm -rf pkgsrc and try again.

    mkdir -p /usr/pkgsrc
    cd /usr/pkgsrc && git init
    warning: templates not found /usr/pkg/share/git-core/templates
    Initialized empty Git repository in /usr/pkgsrc/.git/
    ...

    Wait a while, and you’ll have a working pkgsrc tree. From here, you can bootstrap pkgsrc:

    # cd /usr/pkgsrc/bootstrap
    # ./bootstrap
    # ./cleanup

    This gets you /usr/pkg/sbin/pkg_add.

    At this point, I consider my jail complete. While it doesn’t have all the third-party programs I need, I can now easily install them from within the jail, either from pkgsrc or with pkg_radd.