Small server IPv4/IPv6 pf.conf

I’m deploying IPv6 for my employer. While getting corporate servers up on IPv6 is nice and all, of course I put priority on my own personal Web server.

Just because IPv6 is still populated mostly by early adopters, doesn’t mean we can neglect basic system security. That means that the server needs a packet filter for both IPv4 and IPv6. PF supports filtering both protocols in one ruleset.

The following is a unified IPv4/IPv6 PF ruleset for a small server. It:

  • Allows all traffic from management addresses
  • Allows all ICMP and ICMPv6 traffic
  • Allows traffic to specific ports where we choose to provide service — in this case, SMTP, DNS, and HTTP.
  • Blocks everything else.

    Fill in the IPv4 and IPv6 external addresses, define your interface, adjust the permitted services to match your environment, and you’re ready to go.

    ext_if="em0"
    ext_addr="{192.0.2.40, 192.0.2.41}"
    ext_v6="2001:db8:0:12::2"

    table <mgmt_hosts> const {172.16.0.0/24, 172.16.5.0/24}
    table <v6_mgmt_hosts> const {2001:db8:1:4::2}

    set block-policy return
    set loginterface $ext_if
    set skip on lo0

    scrub in all no-df

    block in all

    pass in on $ext_if proto icmp all
    pass in on $ext_if proto icmp6 all

    pass in on $ext_if inet from <mgmt_hosts>
    pass in on $ext_if inet6 from <v6_mgmt_hosts>

    pass out on $ext_if inet from $ext_addr to any
    pass out on $ext_if inet6 from $ext_v6 to any

    #services we permit
    pass in on $ext_if proto tcp from any to $ext_addr port {25,53,80}
    pass in on $ext_if proto udp from any to $ext_addr port 53
    pass in on $ext_if inet6 proto tcp from any to $ext_v6 port {53,80}
    pass in on $ext_if inet6 proto udp from any to $ext_v6 port 53

  • Book Promotion Methods and Effectiveness

    Lots of people claim to have “the secret” to promoting books. After all, your book is awesome, isn’t it? The problem must be that you aren’t promoting it. You can attend workshops, courses, and buy books that all proclaim The Secret.

    For SSH Mastery, I’m responsible for all of the promotion. I’ve followed the usual advice: I have a blog, I have Twitter, I have a fan page on Facebook, and so on. You can stalk me through any method you choose. I also have real-time access to sales data from Amazon, Smashwords, and Barnes & Noble.

    For this example I’m going to use Amazon data, because Amazon provides very nice graphs through Author Central. If you’re an author and you are not on Author Central, get on it NOW. (Then check it only once a month or so.)

    Here’s the graph for Kindle sales of SSH Mastery, since its release.

    from Author Central

    Note that Amazon’s sales ranking algorithms are rather mushy. More than once, according to Author Central, this book has cracked the #10,000 limit. Most of the time, Amazon retroactively changes that. The peaks shown are generally within a day or two of when the sale took place, however. While this is undesirable, it’s better detail than the quarterly or biannual resolution you get from a publisher.

    So, what have I done to promote SSH Mastery? I’ve blogged it. I blog every time it gets a mention in the press. The blog goes into Twitter and Facebook. I also comment on the book via Twitter.

    I also thank people when they say they buy the book or when they review. Even when they don’t like the book. (This book has received all five-star reviews, but more on that later.) Being known as a nice guy, or at least a polite one, or at least “not a screaming jackass,” has publicity value all its own.

    So, how does all this impact book sales?

    There’s been four reviews or “general public statements about the book that might not be exactly a review” that I know of:

    Peter Hansteen, 22 Jan 2012
    Justin Sherrill, 2 Feb 2012
    Alexander Leidinger, 3 Feb 2012
    Richard Bejtlich, 6 Mar 2012

    (Wow. I forgot to blog Bejtlich’s review. He is my biggest fan and a staunch advocate of my work, and I completely failed to blog his review. I done him wrong. I’m a doofus.)

    There’s also been good reviews posted on all of the ebook sites. I don’t appreciate them any less, but those reviews only help when people go to the retailer’s site. That means that they already know the book exists.

    Compare those dates to the spikes on the graph. Keep in mind the mushiness of the Amazon ranking algorithm. You’ll see that the spikes roughly match up with the reviews. (If I had taken snapshots of this graph on the actual day and the day after the review, the spikes would be much more apparent, but I didn’t think of that until today. Yep, I’m a doofus.)

    There’s another small spike around 20 February that I have no explanation for. I don’t know why it’s there, but I’m glad to see it.

    The initial high sales came from the folks who follow me via my blog, Facebook, Twitter, or some other promotional medium. I was thrilled to see fans buying my work, and of course I appreciate every sale. But that surge didn’t last long, and it’s not enough to sustain a career.

    But reviews expose the book to entirely new audiences. Reviewers are force multipliers. Nurture them.

    In my case, the reviews are all positive. If I really wanted to do an experiment to test the “reviews are the only effective publicity” hypothesis, I’d write a lousy book and see how those reviews impacted sales. But my authorial pride outweighs my scientific curiosity.

    So, the most effective promotion tool? Reviews.

    Other than that: Shut up and write. No, quit jabbering and babbling. Shut up and write. Those words you’re going to use on a comment on this blog post? Put that energy into making your book awesome.

    Shut up and write.

    SolusVM KVM offline migration with shared storage

    I’m building a new virtualization cloud with SolusVM, KVM, and a bit of Xen (to make use of older hardware). Each machine has its own hard disk, but it only holds the local operating system. All virtual machines reside on cheap iSCSI storage, so I can easily migrate VMs from one compute node to another. The goal being, of course, to separate service failures from hardware failures. (I still have to deal with possible storage failures, of course, but hot-swap hard drive arrays reduce my risk somewhat.)

    SolusVM provides a nice front end to the whole Linux virtualization tangle. It does exactly what it claims, and at a reasonable price. I’m happy to pay someone a couple bucks a year per physical server to give me a non-sucky cloud front end that Just Works. One feature that it lacks is live migration for KVM and Xen hosts. Live VM failover is nice, but not essential for my purposes. As part of our Redundant Array of Inexpensive Crap strategy, I cluster VMs as well as physical servers: multiple mail servers, multiple DNS servers, and so on.

    While there’s documentation on how to cold-migrate Xen VMs, there’s no documentation on how to migrate a KVM VM from one node to another, however. Let alone how to do this with shared storage. But the forum says that the Xen method should work with KVM. Let’s try it and see what happens!

    The Xen page talks about replicating the LVM container on the new node. With shared storage, you can skip this step; I defined my SolusVM groups based on the iSCSI device they’re attached to. I imagine the same migration process would work with unshared storage, if you duplicated the disk data first.

    Go into the SolusVM GUI and note the VM number and the node number. For my test, want to move VM 2 onto node 4. Log onto the master server, become root, and run:

    # cd /scripts
    # ./vm-migrate 2 4
    Virtual server information updated!
    #

    I then tried to start the VM via the GUI, and it wouldn’t boot. Log onto the compute node to find out why. Any time I have a virtualization problem involving multiple pieces of hardware, I check /var/log/libvirt/libvirtd.log. Starting the virtual machine generated this log message:

    14:36:13.417: 1443: error : qemuMonitorOpenUnix:290 : failed to connect to monitor socket: No such process
    14:36:13.417: 1443: error : qemuProcessWaitForMonitor:1289 : internal error process exited while connecting to monitor: inet_listen_opts: bind(ipv4,0.0.0.0,5901): Address already in use
    inet_listen_opts: FAILED

    The KVM instance could not use port 5901, because something else was using it. KVM uses VNC to offer console access, and attaches to a port above 5900. Machine number one’s console is on VNC on port 5901, machine number two on port 5902, and so on.

    The migrate-vm script didn’t change the console port. I went into the VM entry, changed the port by hand, and brought up the machine without trouble. Annoying, but not insurmountable.

    Hopefully this helps the next sysadmin searching for this topic.

    SSH Mastery print now on Amazon

    You can now buy the print SSH Mastery from Amazon. The print version isn’t yet linked to the ebook version, but that will happen within 1-3 days.

    To my surprise, Amazon has not discounted the book. I don’t know why. I spoke with their author support desk, and was told that some books get discounted and some do not, according to some internal algorithm that only Amazon knows. All of my other books get discounted before publication. I assume that SSH Mastery will be discounted as well, but I have no idea when. (Yes, Amazon has an author support desk. From clicking “Call me” to resolution and hanging up the phone, total time 2 minutes 29 seconds. I am impressed.)

    For those who are interested in the numbers behind the print version:

  • If you buy from OpenBSD, I make zero. The proceeds go to support OpenBSD/OpenSSH development. I am perfectly content with this.
  • If you buy from Amazon, I make about the same as I would if you bought the ebook.
  • If you buy from my CreateSpace store, I make about twice as much money as an Amazon purchase. But there’s no Amazon Prime, no free shipping when combined with other purchase, and no discount, ever.

    Some thoughts on the CreateSpace store, while I’m at it:

    CreateSpace gives me the store automatically, for free, so I link to it; otherwise, I wouldn’t bother setting it up. Purchases from here are “giving the author extra money because you want to.” And to my surprise, someone actually bought one there, so: thank you, anonymous buyer. (Note that Amazon/CreateSpace also makes more money when you buy from the CS estore.)

    In the event that people actually start buying from the CreateSpace estore, I’d probably set up a PayPal tip jar, so that those folks who want to give me extra money can do so and still get free shipping and/or discounts. It’s a step that I’ve avoided, but if people actually want overpay me for work I honestly own, who am I to argue?

  • SSH Mastery Print Here

    Today, I received a good proof of SSH Mastery. All of the errors I know of are fixed.

    I’ve ordered 200 books for the OpenBSD Project. Those books should be manufactured this weekend and delivered next week. (For the record, the OpenBSD guys have been a pleasure to deal with.)

    Amazon should have the books available in ten days or so, Barnes & Noble and other Ingram-connected bookstores a while afterwards.

    If you absolutely must have the print book now, you can order it through my CreateSpace store. Of all the ways the book is available, I make the greatest profit on books sold through the CS store. (It also charges list price, so I’m not going to push it on people. Only order from there if you want to give me extra money.)

    If you want to pay full price, you can order it from OpenBSD. I give them the books at cost, and the rest benefits OpenBSD and OpenSSH.

    Remote Web Browsing via OpenSSH and PuTTY

    I’m installing SolusVM as a virtualization management system. It lets you manage your private cloud via a Web browser, set up resellers, and so on. When you first log in, the administrative interface locks itself down so that you can only log in from one IP address, in a sort of implicit whitelist. You must explicitly add other addresses. That’s fine, even reasonable. I had three address ranges to add: my office, the headquarters, and Fearless Leader’s office. So I went into the management interface and explicitly added the headquarters’ addresses.

    And I was locked out of the management interface. Apparently the explicit whitelist permitting HQ overwrote the implicit whitelist permitting my workstation.

    I could have opened a ticket with SolusVM and admitted that I’d ignorantly locked myself out. But I don’t like interacting with vendors. I could have driven into the office, but that would involve changing out of my bathrobe. That left logging into the management workstation via a web browser from headquarters. I’m not going to talk one of my coworkers through it if I can avoid it.

    Instead, I used SSH dynamic forwarding to connect to the SolusVM head node from an IP address at headquarters.

    You can do this with an OpenSSH server and either a PuTTY or OpenSSH client. I chose to use PuTTY because that was the computer on the couch with me. I have several OpenSSH servers at headquarters.

    Open a new PuTTY session. Enter the host, username, and server port as normal. Before opening the session, go to the left-hand side of the screen and select SSH -> Tunnels. Enter a “Source port” of 9999. Near the bottom, select “Dynamic.” Now open your SSH connection.

    You now have a SOCKS proxy running on your computer. All traffic sent to port 9999 is sent over your SSH session. Your SSH server connects you to the Internet.

    Go to your Web browser’s connection settings. In firefox, it’s Tools->Options->Advanced. Select the Network tab, then Settings. Select Manual proxy settings, then enter a SOCKS host of 127.0.0.1 port 9999. Select the SOCKS5 button. Exit the menus, hitting OK all the way back.

    Now your Web browser connects to the Internet via the SOCKS proxy running on your computer. You’re browsing the Web from the IP address of your SSH server.

    This is much faster than remote browsing options such as Remote Desktop or forwarding X11 over SSH. And it let me log into my SolusVM console without having to communicate with another human being, so everybody wins.

    Of course, you could learn about this sort of trick and more in my new SSH book.

    Print pre-order for “SSH Mastery” now available

    The OpenBSD folks have made pre-orders available for SSH Mastery.

    I’m giving the books to the OpenBSD Project at cost, and they’re selling them at list as a fund-raiser. I make nothing on books ordered through OpenBSD, but the money will go to further OpenBSD, OpenSSH, PF, and who knows what else. I’m going to imagine that the money will support the annual developer barbeque, as buying beer is more fun than buying hardware.

    I will make a decent profit on books sold through Amazon, Barnes & Noble, and brick-and-mortar stores. (Yes, I can get books into brick-and-mortar stores, with a bit of luck, a smidgeon of effort, and just a few ounces of blackmail.) I figure it’ll even out; the general public will order from Amazon.

    When will the books arrive? Once the proofs are approved, I’ll order the OpenBSD books before I approve sale through Amazon. I don’t know when that is yet, but believe me, I want this book out there.

    So, if I make nothing on this, why do it? It’s a good way to support a widely-used project that has written software that makes my job easier. So order yours now!

    Plus, I write about open-source software projects, preferably BSD-based ones. This means that I have to maintain good relationships with those projects. Not only am I a geek, I work at home. My usual social interactions consist of ignoring the threatening emails from the boss, the pet rats asking for treats, and the nagging phone calls from the social worker asking if I’ve taken my meds today. As you might guess, maintaining good relationships is not my strong point. If you have trouble keeping friends, try bribery. I hear it works a treat.

    “SSH Mastery” print status

    Thursday, I received the first print proofs of SSH Mastery.

    Initial impressions: the print version looks good. The photo doesn’t do the nice glossy cover and bright colors justice, but:

    After careful inspection, though, I found a few errors. The interior images are not as sharp as I would like. Some of these I can’t really help: printouts of screenshots look like crap. But I’m more concerned about pixellation in diagrams and images.

    Plus, there’s an extraneous colon on the back cover.

    If I want this book to be taken seriously, it needs to look like a real book. So I’m getting the layout corrected and running another round of proofs. I suspect I’ll get the new proofs late next week. If those are OK, I will be able to open orders.

    I’ve been working with Austin from the OpenBSD group to get their ordering up soon.

    January “SSH Mastery” sales, by the numbers

    I’ve promised several authors to share the results of my self-publishing experiment with SSH Mastery. I don’t have complete data yet, but I do have sales numbers for January from Amazon, B&N, and Smashwords.

    Some caveats here:

  • This includes only “SSH Mastery.” I have removed my fiction from the totals. (Fiction sales are considerably lower, but growing.)
  • February’s initial results are much lower than January’s. January’s sales are obviously to my “hard-core fans” and people close to the community.
  • My expenses aren’t yet totaled, as the print version isn’t available yet. I expect them to come in at roughly $3000. The majority of this expense is the class I took to learn how to self-pub properly, which is a one-time expenditure. A business person would argue that this expense doesn’t count, as it should be spread across multiple books. But as an author running a test project, I have to count this expense somewhere, so it’s attached to this book. Also, these expenses are only cash out of pocket; they assume my time is free.
  • On a related note, an accountant would probably find my reasoning naive. I already have two careers (writer and engineer), I’m not going to add a third.
  • I’m not going to regularly report sales numbers to the general public. I will say when I break even.
  • Before I published, I expected to make most of my sales through Amazon, then Barnes & Noble, and then Smashwords last. I had no idea of how many sales I would make, but I figured they’d be in that order. Let’s see how this compared to reality.

  • Amazon Kindle: 123 books sold (92 US, 12 UK, 14 DE, 2 FR, 1 IT, 2 ES), for a total of $810.27 USD at today’s exchange rates. (Amazon reports European royalties in euros or pounds.)
  • Barnes & Noble: 4 books sold, for $25.96 USD.
  • Smashwords: 76 books sold, for $607.65
  • Total: 202 books sold, for a total of $1443.88
  • Some interesting things here:

    I’m shocked at how low Barnes & Noble sales are. The book was available on B&N a couple days after Smashwords and Amazon, mainly because getting the book through B&N’s internal systems took longer. Apparently my readers don’t use the Nook.

    Smashwords sales as a proportion of total sales is much higher than I expected.

    Averaging the royalty per book is also interesting.

    Amazon: $6.58/sale
    Smashwords: $7.99/sale
    B&N: $6.49/sale

    You’ll hear lots of commentary about how Amazon offers a 70% royalty on ebooks. This has all sorts of exceptions and exclusions, where you’ll only get a 35% royalty. My effective average royalty at Amazon is about 65%, so that’s not bad.

    Barnes & Noble, at just under 65%, offers the worst net royalty.

    Smashwords: I love you. Just saying. My core audience really wants the book as PDF, and Smashwords offers PDF, epub, mobi, and all the other big ebook formats with one purchase. And they pay me the highest per-sale royalty. I still haven’t gotten the book through Meatgrinder into the other channels they feed, such as iBooks and Kobo. Once that happens, I expect to see their percentage rise. I wouldn’t be surprised if they overtake Amazon.

    The real lesson is: sell your work through all available channels. You can’t tell who will buy what from where.

    Installing WHMCS on FreeBSD 9.0-RELEASE

    Or, if you prefer: “WHMCS versus PHP.” Blogged for the next sysadmin searching Google.

    $DAYJOB recently acquired WHMCS to help automate virtual server provisioning, billing, and so on. According to everything I’ve read, WHMCS runs just fine on FreeBSD, so I installed the prerequisites on a 9.0-i386 machine. As with any server for PHP-based Web sites, I verified that the server processed PHP with a simple phpinfo() page. I then grabbed the WHMCS tarball (no link, you must be a customer, sorry), extracted it into the directory, ran the setup program, fed in the database information and license key…

    ..and it wouldn’t run. Calling up the app resulted in a blank page. WHMCS provides troubleshooting instructions for this exact circumstance. I enabled the requested debugging, but couldn’t get WHMCS to produce an error. Adding a bogus argument to my phpinfo() test page made an error appear, so I was confident the failure to display an error message wasn’t a server configuration problem.

    This comes down to FreeBSD’s PHP packaging.

    When you install PHP on the popular varieties of Linux, you generally get a whole slew of PHP extensions with it. BSD-based systems only install exactly what you ask for: if you want PHP but don’t request any extensions, you won’t get any extensions.

    I agree with this approach. Every piece of installed software needs patching and updating. Every piece of installed software is a potential attack vector. If I don’t need a piece of software, I don’t want it on my server.

    WHMCS doesn’t list all of the required extensions. They assume you have a kitchen-sink PHP install. After some reading and research, I found that WHMCS runs fine with the following PHP modules and extensions installed. I’ve included the version numbers for reference, but you should be able to just pkg_add -r all of these by name.

    php5-5.3.8
    php5-bz2-5.3.8
    php5-ctype-5.3.8
    php5-curl-5.3.8
    php5-dom-5.3.8
    php5-extensions-1.6
    php5-filter-5.3.8
    php5-gd-5.3.8
    php5-hash-5.3.8
    php5-iconv-5.3.8
    php5-json-5.3.8
    php5-ldap-5.3.8
    php5-mysql-5.3.8
    php5-openssl-5.3.8
    php5-pdo-5.3.8
    php5-pdo_sqlite-5.3.8
    php5-phar-5.3.8
    php5-posix-5.3.8
    php5-session-5.3.8
    php5-simplexml-5.3.8
    php5-tokenizer-5.3.8
    php5-xml-5.3.8
    php5-xmlreader-5.3.8
    php5-xmlwriter-5.3.8
    php5-zip-5.3.8
    php5-zlib-5.3.8

    I’d really like to trim this down to only what is strictly necessary to run WHMCS, but that information doesn’t seem to be available. I could methodically remove and reinstall extensions to see when WHMCS breaks, but I have better things to do than debug missing docs for a commercial PHP app.

    On the plus side: now that WHMCS is installed, it’s really slick. I’m looking forward to using it. Actually, I’m looking forward to having other people use it for me, so I can do more interesting things than provision servers, accounts, and billing.