Jan 2015 Status & Next Projects

Welcome to 21 January 2015. Here’s the news.

I finished the first draft of Tarsnap Mastery yesterday. Today I went through it one more time, then shipped it off to Colin Percival for his comments on the last few chapters. Once I have his corrections, I’ll solicit reviewers.

Networking for Systems Administrators is at the copyeditor. It’s due back Friday. I expected this book to be pretty easy, but the tech reviewers savaged it. The end result will be a much better book, but it still wasn’t much fun and took longer to repair than I expected. With any luck, though, I’ll be able to get the electronic version out before the end of January and print in mid-February.

These two books are not available for pre-order through the Tilted Windmill Press web site. Books that I offer direct pre-orders on have done much more poorly than books without direct pre-orders. (Part of that might be topics, of course.) Other authors tell me that Amazon uses a book’s initial sales velocity to compute a book visibility to other buyers. More than 90% of my TWP book sales come from Amazon, so I care what happens on that platform.

The sequel to Immortal Clay is rolling along.

Now that Tarsnap Mastery and Networking for System Administrators are in a lull, I’ve started seriously pulling material together and filling in the outlines for my next FreeBSD Mastery books:

  • Jails (#fmjail)
  • ZFS (#fmzfs)
  • Specialty Filesystems (#fmspf, because #fmsf is taken)

    If you watched my most recent BSDNow interview, this is not a surprise.

    The books will assume you know what’s in FM: Storage Essentials, and there is a certain amount of interconnection between all three. For example, to use jails you should know about devfs and unionfs. The Jails book will include the incantations to perform the devfs and unionfs tasks needed for jails, but the explanations for them will be in the Specialty Filesystems book. Similarly, the Jails book will have ZFS rituals in it, but the ZFS book will have the knowledge behind those rituals.

    So, if you know some of these systems but not all you only need buy exactly what you need.

    I might split the Specialty Filesystems book into two parts, one for local filesystems and one for networked filesystems, depending on how long the book gets and the final content. My goal for these books is to make them about 30,000-40,000 words.[1] FMSE was 45,000 words, and N4SA is about that long. The sudo and Tarsnap books are closer to 30K, while SSH is right around 35K.

    I expect that once I finish them, I’ll have a bundle at Tilted Windmill Press much like the existing Security Bundle.

    When will these be finished? I really want to take the finished print books to BSDCan in June.

    [1] Why restrict book length? While the various ebook platforms do not restrict how long books can be, they do restrict how much I can charge for them. Most traditional publishers do not have that restriction. I must stagger roughly around that fuzzy intersection between “give good value,” “include what the reader needs,” and “can’t pay the mortgage.”

    Note that the on-demand printers do limit the size of print books. The bindings at the larger size books are not great. I refuse to release shoddy print books.

  • Two “FreeBSD Mastery: Storage Essentials” reviews

    Two new reviews on FreeBSD Mastery: Storage Essentials yesterday.

    First, from Justin Sherrill, on DragonflyBSD Digest.

    Then on Slashdot, which got featured on Reddit, from Saint Aardvark the Carpeted. (The nice thing about the Internet is, even a carpeted aardvark can make his mark on it.)

    I’d like to thank both gentlemen for taking the time to review my latest. Reviews from disinterested third parties can make or break an author.

    BSDCan 2015 Call for Papers Ending Soon

    I just submitted a talk and a tutorial to BSDCan 2015.

    If you’ve done anything in BSD technology over the last year more sophisticated than “maintained respiration and blood flow,” you should submit as well.

    As a committee member, I can say we are always looking for unusual work. Yes, we all know that Henning Brauer is working on OpenBSD, and Luigi Rizzo is doing something clever with the network stack, and Max Stucchi is pimping IPv6. That’s all well and good, but we always want something outside our usual suspects. You’re the network administrator of a salt mine, or the frozen North, or an oil rig? You’ve wedged BSD into an alarm clock? Submit! Tell us about it!

    Otherwise, they’re gonna let that Lucas character talk. Again. Nobody wants that.

    SMLR on “FreeBSD Mastery: Storage Essentials”

    The Sunday Morning Linux Review folks have a review of FreeBSD Mastery: Storage Essentials in show 141. The review starts at about 39:30, but the whole show is worth listening to. As always.

    For my own reference, here’s a couple key quotes that I’ll probably use for marketing later. (I either write them down here, or have to go listen to the show again when digging up blurb quotes later.)

    “Lucas lays a solid foundation about disks.”

    “The devil is in the details, and the details are in the book.”

    And when it comes to slices versus partitions: Mary is right. Listen to her, guys.

    “FreeBSD Mastery: Storage Essentials” print available!

    You can now get FreeBSD Mastery: Storage Essentials in print on Amazon.

    If you buy the print from Amazon, you can get the Kindle version for $2.99. Sadly, that’s the closest thing to a proper print/ebook combo I’ve been able to do.

    For completeness’ sake: you can also buy it directly from my CreateSpace store. As sales go, that’s where I make the most money. It’s also the most expensive version. If you want to pay extra so I make more I won’t object, but I will suggest you avoid the middleman and go straight to me.

    It will appear in places like Powell’s in the coming weeks.

    Now to finish “Networking for Systems Administrators,” complete the design on the next tech book, finish the outline for the Immortal Clay sequel, and finish outlining the intertwined morass that is my next three FreeBSD Mastery books.

    FreeBSD fetch(1) broken on SSL links?

    I went to download Tarsnap on a FreeBSD 10.0-p12 machine, and got hit with this error:

    # fetch https://www.tarsnap.com/download/tarsnap-autoconf-1.0.35.tgz
    Certificate verification failed for /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
    34380830376:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1179:
    fetch: https://www.tarsnap.com/download/tarsnap-autoconf-1.0.35.tgz: Authentication error

    Looking at the last line of the error, you might think that Colin password-protected the Tarsnap source code. This would be extremely daft on his part, so I read on. But actually reading the message tells me that fetch(1) died because it couldn’t verify the Comodo RSA cert used on the Tarsnap web site.

    Comodo has been around a long time. Why would their cert be invalid?

    Second thought: Colin’s been hacked!

    But no.

    The third thought is the charm. Turns out that fetch in FreeBSD 10 validates SSL certificates–but doesn’t ship with an SSL root certificate! So, Colin hasn’t gone daft, or been hacked… but someone in the FreeBSD crew definitely increased my astonishment!

    I installed the ca_root_cert package and created a symlink for fetch.

    # ln -s /usr/local/share/certs/ca-root-nss.crt /etc/ssl/cert.pem

    Fetch now worked as I expected.

    It does seem that if you’re going to validate SSL certs, you should either have a decent root cert bundle installed or print a helpful error message.

    FreeBSD Mastery: Storage Essentials at printer

    Last night I received the print proofs of the new FreeBSD book.

    fmse proofs

    I found two errors: a missing tab in a footnote, and an extra page before the index. I’ve fixed those, double-checked the file, and sent it to the printer. It should be available in print in the next week.

    I’ll have four copies at next week’s sudo talk at mug.org. Show up and you might be able to bribe me to get a copy of one of these very exclusive rare, authentic, original, limited edition books.

    I’m very happy with the look of the final print. The cover is spectacular. Tech book usually have bland covers, but I decided to try something a little different. I’ll have custom covers like this on at least two other books, FM: ZFS and FM: Specialty Filesystems. (And before you ask: no, no release date yet, except “2015.”)

    Three books should give me enough data to see if there’s a return on investment for fancy illustrated covers on tech books. If the book doesn’t sell well enough, I’ll fall back to more traditional tech book covers based on photographs.

    Book Review: Book of PF, 3rd Edition

    No Starch Press was kind enough to send me a review copy of the new 3rd edition of Peter Hansteen’s Book of PF. The first two editions are the standard reference work on the OpenBSD packet filter, and this is a topic I’ve written about in some depth before, so I’m fully prepared to eviscerate Hansteen if he screwed up.

    Third edition, huh? So how does this stack up?

    Let’s get the obvious out of the way. All three editions have yellow covers, but the first edition had blue trim; the second edition, pumpkin; the third, a kind of fern green.

    Once you open the cover, you find that the third edition works much like the earlier editions, starting with a simple packet filter setup and building upon it. Building upon success is perhaps the best education technique, and it fits the topic quite well.

    Structurally, BoPF3 is very similar to the earlier editions. CARP and redundancy now gets its own chapter, which is a welcome addition.

    The real meat of this book is in the examples, tutorials, and explanations. PF has changed since the second edition, notably with more flexible traffic management and some syntax changes. Not all BSDs have remained synchronized with OpenBSD’s PF, so he has the unenviable job of documenting the differences between OpenBSD, NetBSD, and FreeBSD. He’s done an excellent job of this by combining information where appropriate, but breaking out some topics by operating system. For example, if a topic needs a sysctl, he lists them for each operating system. When a topic requires more in-depth explanation, such as traffic prioritization, he breaks out OpenBSD’s new priority system into one section and FreeBSD/NetBSD’s older altq prioritization scheme in another. This makes it very easy to find what you’re looking for. This book teaches you how to use PF to filter packets just as well as a million dollar appliance, with more insight and control.

    One impressive thing is that this book is very clear. Giving a section a title like “Things You Can Tweak and What You Probably Should Leave Alone” gives you very definite ideas about what’s in this part of the book. Hansteen explicitly describes how PF works. He also discusses what happens when the real world impacts your firewall. He hasn’t just got PF up and running in his lab: he actually uses this stuff in the real world, with all its malformed packets and stupid protocol implementations and worse protocols, and keeps services running despite all that.

    Complaints with this edition?

    As with the earlier editions, the footnotes contain actual facts. It’s like Hansteen wants you to be able to go look up actual sources to verify what he says, instead of requiring us to trust him. As a writer, I prefer assuming blind faith and unyielding obedience from my readers.

    If you use PF on any platform, buy this book. You can get it from Amazon (of course), and also get a combined print/DRM-free ebook deal direct from No Starch Press.


    This ends the actual review. But someone is going to ask a couple things in the comments, so I’m jumping ahead of them here.

    First: my general thoughts on authors writing reviews.

    Second: Why hasn’t FreeBSD imported the latest PF?

    The FreeBSD and OpenBSD network stacks have massively diverged in the last twenty years. OpenBSD’s kernel uses the Big Giant Lock model. FreeBSD’s kernel is much more finely locked, and the network stack can be in multiple CPU cores simultaneously. Despite their common heritage and licenses, FreeBSD and OpenBSD are different operating systems. They have different use cases. They are designed for different uses. They target different hardware.

    FreeBSD’s previous PF import required a lot of work to make it fit its network stack. The FreeBSD Foundation invested a fair chunk of change in thrashing PF in the network test cluster and on high-performance customers so that it didn’t slow down the network stack. (It’s not that OpenBSD is slow, it’s just designed differently than FreeBSD.)

    A “new import” is not trivial.

    FreeBSD has a flexible firewall system, however. A new PF could be imported as, say, pf56.ko, without impacting the older PF import. You could use mailwrapper-like functionality to transparently assign the proper userland programs to the PF version in use. This can be done.

    Nobody has done the work.

    I suggest you get coding.

    FreeBSD “Working copy ‘/usr/src’ locked.”

    Poul-Henning Kamp is working with me on some GBDE fixes. Which means he sends me patches and says “Here, try this,” along with very valuable exposition on how GBDE works and the threat model it applies to. This means I’m updating frequently.

    My usual update process is:

    # cd /usr/src
    # make update && make -j8 buildworld && make -j8 kernel && reboot

    Half a second after typing this, I realized I’d forgotten to apply PHK’s latest patch. I hit CTRL-C during the make update before building an unsuitable userland. I need to do the source update, apply the patch, and then build everything.

    # make update
    --------------------------------------------------------------
    >>> Updating /usr/src using Subversion
    --------------------------------------------------------------
    svn: E155004: Run 'svn cleanup' to remove locks (type 'svn help cleanup' for det
    ails)
    svn: E155004: Working copy '/usr/src' locked.
    svn: E155004: '/usr/src' is already locked.
    *** Error code 1

    Stop.
    make[1]: stopped in /usr/src
    *** Error code 1

    Fine. svn cleanup it is. But wait–I don’t have svn installed!

    As a matter of principle, I don’t want to install svn on this test box. svnlite is what FreeBSD offers to users, so it should be able to handle everything.

    The good news is, svnlite also have a cleanup feature.

    # svnlite cleanup
    # make update

    And the update proceeds as I would hope.

    Is this worth a PR to get the error message changed? Dunno. What do you think?

    Now all I must do is master all the GBDE wisdom PHK dumped in my brain…