FreeBSD Ports Annoyance

Ivan Voras’ article on FreeBSD’s pkgng prompted me to try pkgng. pkgng works exactly as advertised, with a couple of minor annoyances. But this brought to head a problem with FreeBSD that I’ve had for a while. I’ve talked to various ports guys about it over the years. It’s an engineering problem that’s begging for someone to solve.

Before folks in other Unix-like operating systems start snickering at “Lucas turning like a rabid dog on the community he came from”: you guys have your own problems. Go deal with them.

First, some context:

When the Ports Tree started it was pretty durn good. It might have started life as a temporary workaround, but it came out all right. Ports was the envy of many operating systems, and it got dragged into other BSDs and copied elsewhere. And in many ways, it’s actually aged pretty well. I run all three “Big BSDs” and a DragonFly box. I administer CentOS and Ubuntu machines. I have a couple of OpenSolaris servers slated to be replaced with FreeNAS. I feel qualified to offer an opinion on how things compare. FreeBSD trumpets its performance as a network operating system, as well it should. It passes packets like “whoa.” It makes lots of complicated stuff easy.

But some simple tasks are harder than they should be.

I’m standing up a new server to replace my old Web server. The old server originally ran 6-current, and has been updated to -current every few months. Now and then, Apache hangs up. I’m fairly confident that some lingering cruft from a previous aeon is responsible, and while I could exhaustively search the old server for the problem, I’d rather send it off to a well-deserved retirement and put some new hardware in play.

So, I have a virgin server. I want sendmail, mutt, and a WordPress server. (Your choices might differ, and that’s fine.) I don’t care which Web server WordPress runs on, anything vaguely modern would do. I’d like to install all software from packages, as I don’t have the time or energy to build an artisan server. Ideally, I won’t even need to install the Ports Collection. This is an extremely common set of software.

I can do this easily on OpenBSD, because OpenBSD has One True Web Server. It’s Apache 1.3, yes, which makes my bowels churn a bit, but the OpenBSD Project stands behind it, so: fine. I can do this with CentOS or Ubuntu or OpenSolaris — I have to install new repositories, which brings up all kinds of trust issues, but when you run those operating systems you decide to evaluate those risks on an ongoing basis. Fine. (I must confess, I’ve never installed a Web server on my NetBSD palmtop. That’s too geeky even for me.)

But I just can’t do this with FreeBSD.

The Ports System is massively flexible. If I’m willing to build from source, I can configure my system in dang near any way I like. That’s an awesome feature. It really is. The Apache 2.2 port has about eighty configuration knobs. I can tweak them all. I can build Apache exactly as I want and be highly confident that it will Just Work.

This is amazing.

But I can’t install a WordPress server with precompiled packages.

The problem is in the PHP port. PHP doesn’t know which Web server you’re running. The Ports System gives you the flexibility to build PHP with or without the Apache module. You can build with or without every dang module in the world. The PHP port doesn’t tie your hands.

But without these — comparatively tiny — bits of precompiled glue, I can’t make FreeBSD Just Work.

And it’s not just WordPress. I run an LDAP server at work. Apache 2.2 can be compiled to include the OpenLDAP module. But I cannot install this common corporate configuration from packages, I must compile the software myself. If I want OpenLDAP and Postgres, well, that’s a different build. And if an application absolutely requires MySQL, it’s back to the Ports Tree.

Multiply this by the thousands of combinations of options in over 23,000 ports, and… wow. Building a package for every possible combination would take a long time.

I can — and do — build my own package repositories. But this is an ongoing annoyance. And if I need an Apache 2.2 server that supports OpenLDAP, MySQL, and Postgres, I’m back to building it myself.

The Ports System makes the difficult manageable. I’d much rather maintain my own FreeBSD package repo than a CentOS or Ubuntu/Debian repo. But I really shouldn’t have to.

I’m not saying “copy Linux.” By all that’s sacred, no. NO. But they’ve solved this problem without building umpteen bajillion packages. We have some really brilliant people in the community. Surely there’s a realistic solution to this?

I pimp BAMP (BSD, Apache, Mysql, PHP) as an alternative to LAMP. This particular example of this issue stops those efforts cold.

Just in case you’re one of the brilliant people in our community, and this motivates you to do something, as a sysadmin who wants to run a lot more FreeBSD, here’s what I’d like to see:

  • Don’t rely on external package repositories. Every time I need to add a new repository to make some Linux package work, my blood turns to icewater and the voices in the back of my head start gibbering again. FreeBSD has a centralized model. I trust FreeBSD’s central package builds; they’re highly consistent, and if they get compromised, I will hear about it.
  • I’d like to be able to set variables in, say, /etc/make.conf or /usr/local/etc/pkg.conf that say what combination of software I use. Say, WEBSERVER=apache22 and DATABASE=postgres91. There are ports that will configure the ports-building process for me, such as ports-mgmt/portconf; I’d like something similar for packages.
  • When I install a new server, I want to be able to copy the config file in, run pkg add wordpress, and have the package system say “Aha! The Boss wants to run this webserver, and this database, but the software package doesn’t support his chosen database, so I’ll install MySQL anyway.” It should then just Do The Right Thing.
  • Why don’t I fix this? I’m a sysadmin, not a programmer. Hogs, when they look at my code, vomit. My spare time is spent writing, most often in support of the various BSD communities. I could learn to fix it, but I believe that others are better qualified to do it first. But those most likely to be able to fix it are also those most likely to just build the software themselves.

    We run operating systems to use applications. (If that wasn’t true, I would have truly mastered my Sinclair ZX80 by now.) The new user, confronted with a requirement to install the Ports Tree to build a simple PHP webserver, might well say “ick” and install something else instead. It’s a serious barrier to entry.

    So, if you’re looking for a new problem to solve, here it is.

    Writing New Editions

    This post is, “how is the new edition of Absolute OpenBSD coming along?” with a bit of musing on the craft of writing a second edition added in.

    I’m always shocked by the number of systems administrators ignorant of networking basics. I don’t care that they don’t know how to choose between BGP and OSPF, or that they don’t know what those acronyms stand for. That’s not relevant to most servers. But lots of them don’t know what an IP address is, or how to recognize a valid netmask, or the difference between TCP and UDP, or why there’s an /etc/protocols file. Any sysadmin who doesn’t know these things is still an amateur. My goal in writing a book is to drag people a little closer to professional.

    So, I include a chapter on networking basics in my introductory sysadmin texts, just like I include chapters on user management.

    I have a chapter on IPv4 networking in three published books. I’m writing this same chapter for the fourth time. I can’t just copy-and-paste from earlier editions. First, that would be rude. Second, my understanding of TCP/IP has changed in the last ten years, and that changes how I approach the material.

    But I can use the earlier efforts as models. Some text I can almost reuse, because it’s still the best way I know of to explain the specific topic. This will be the third time I use the dinner table analogy, for example. I still pass it through my brain to my keyboard, however, freeing myself to tweak a few words in the process.

    The most recent IPv4 chapter I wrote was for Absolute FreeBSD. In this incarnation, the chapter included a couple pages of basic binary and hexadecimal math. I looked at this, and thought “Why did I cover this? Doesn’t everybody know it?”

    Then I thought back, and realized that I included those pages because at the time I wrote the book, I spent a fair amount of energy teaching that material to my coworkers.

    I flipped back through the earlier editions of these books. Each book had one or more sections that I included because coworkers didn’t know it.

    At the moment, I’m not responsible for teaching anyone anything. I have no tech minions, and am molding nobody. It’s definitely changed my mind about what topics I cover. I suspect that the new edition of Absolute OpenBSD will contain less basic material. I’m definitely assuming that you know how to do binary and hexadecimal math, for one thing. This leaves room for more advanced topics.

    My conclusion seems to be: if you find the new edition of Absolute OpenBSD moves too fast, I suggest you get a copy of Absolute FreeBSD as well, or reread the one you have.

    (Mind you, the no-minions-to-mold is about to change. After two years of minion-free peace, I have been given a minion to mold. He’s on vacation at the moment, and has no idea what awaits him on his return. I have no desire to ruin his last few days of freedom, so we’re waiting for his first day back to tell him what he’s been sentenced to. The poor bastard.)

    So, where am I on the second edition of Absolute OpenBSD?

    If you want the minutia of my progress, search for the #ao2e hashtag on Twitter. But at a larger level, I’m writing the chapter on IPv4.

    This seems to be about 40% through the first draft of the book. The manuscript has proceeded quickly, now that I’m not moving into a house that needs work to be habitable. I’m hoping that this pace continues.

    I’ve received initial feedback on chapters 1-8 from Henning Brauer. Then the chapters go to Nathan Houle, my editor at No Starch Press, then back to me for corrections and discussion. Then they go to Peter Hansteen for formal technical review. Then back to me for correction. Then copyedit, back to me for correction, page layout, and back to me for correction.

    So, it’s not 40% done. The first draft is the hardest part, however. Doing the math, though, I see that I’ve been through an IPv4 chapter at least twenty times, given all the cycles. No wonder writing it is causing me nausea and chest pain.

    The original outline calls for a book about 400,000 words. For reference, Absolute FreeBSD is close to 300,000 words. This is too dang long. One of my goals is that my books be small enough to read in the bathtub.

    As SSH Mastery was successful, I have a resolution. I think I’ve figured out topics I can extract from the book and publish separately, without damaging the integrity of the book or its usefulness. Not everybody needs to know about, say, OpenBSD’s wireless features, but it’s certainly a topic worth covering. I can do small books on those topics and publish them as an aside, making the content available to interested readers. Assuming that the reader is a competent OpenBSD sysadmin (e.g., they’ve read Absolute OpenBSD 2nd ed or have an equivalent combination of education and experience) will let me do these books almost as easily as if it was integrated into the book. And my initial market research indicates that my readers are amenable to smaller, single-topic books.

    In summary: book is underway. More books coming.

    Cisco radius auth for users and enable

    All authentication on my network (with carefully selected exceptions) should be centralized. This includes router administrative logins via telnet or SSH. My authentication information is in an OpenLDAP 2.4 server. Attaching Cisco gear to an OpenLDAP database is hard. But attaching Cisco gear to RADIUS is pretty easy. But my FreeRADIUS server uses LDAP as its back end, and attaching Cisco gear to RADIUS is pretty easy.

    To have your enable password, you’ll need an LDAP user called $enab15$. Take careful note of how that is spelled: dollar sign, ENAB, the number 15, and another dollar sign. There is no L, and no second E. Add this user to any LDAP groups needed for RADIUS access. This user’s password is your enable secret.

    Create a loopback address for the router.

    interface Loopback0
    description management
    ip address 192.0.2.13 255.255.255.255

    Use this address for all router management functions. If you use an IP for a real interface for management or monitoring, you can have trouble when that interface goes down.

    Then tell the router about your RADIUS servers. Always list multiple RADIUS servers. If you have only one RADIUS server, get a second, preferably on a completely different part of your network.

    ip radius source-interface Loopback0
    radius-server host 192.0.2.253 auth-port 1812 acct-port 1813 key RadiusSecret
    radius-server host 192.0.2.252 auth-port 1812 acct-port 1813 key RadiusSecret

    Create a local enable password and a local user with administrative privileges. These will come into play when your RADIUS servers fail. (Hopefully, they never will. But assuming things will go well sets yourself up for a really bad day.)

    enable secret 5 $1$lnds$LNrkh4d8aoeuY/Q2Akm1k7
    username admin privilege 15 password 7 1D1C1B050B8290E1

    Now attach the radius servers to the authentication system.

    aaa new-model
    aaa authentication login default group radius local
    aaa authentication enable default group radius enable

    With this setup, your Cisco will try radius first, and then fall back to the local authentication file if your RADIUS server does not answer.

    You might also need to attach your virtual terminals to your authentication settings.

    line vty 0 4
    login authentication default
    line vty 5 15
    login authentication default

    You should now have Cisco user names and the enable password synchronized with LDAP, through RADIUS.

    I’m in BSD Magazine

    The July 2012 issue of BSD Magazine has an article by yours truly: freebsd-update as an Intrusion Detection System.

    It also has a code to get you 30% off of Absolute FreeBSD at No Starch Press. If you don’t have your copy of this book, here’s your chance.

    It has other good articles too. None as awe-inspiring as mine, of course, but definitely worth a read.

    Keeping Friends

    I’m heading out to Oregon for Kris Rusch’s short story workshop in a little while. Additionally, I just got my story collection Vicious Redemption out in print. So, what the heck, here’s a story. It’s short enough that I’m not comfortable putting it out as a 99-cent short, but sufficiently solid that it deserves an audience.

    Warning: not for children.

    Keeping Friends

    “I’m trying to decide if I should kill myself now, or wait five minutes.”

    My precognition hadn’t warned me about Tom’s call, but I hadn’t asked it. I pressed the cellphone into my ear hard enough to hurt, trying to compensate for the crackling connection and the passing traffic. “You don’t really want to do that, dude. You still have options. There’s other meds out there.”

    “They won’t work. Nothing works. There’s no hope.”

    I sat on a bench and forced my hands to stay still. My friends are important to me – I have to keep them. If I failed, Tom might kill himself while talking to me. The thought heated my blood until the air felt cool. “Can I come down there?”

    “Don’t bother. I tried a knife, but it wasn’t sharp enough to cut through my neck.”

    Tom had fought clinical depression for two years. Medication had taken the edge off, but not enough that he could return to work. The electroconvulsive therapy his doctor had prescribed required Tom stop taking all his medications. The edge had returned, and he was using it on himself.

    “Are you holding the knife now?” I said.

    “Yes.”

    “Do me a favor. Put it down.”

    “I need to sharpen it. That’ll take both hands.”

    The phone clicked and I heard only silence, then the dial tone.

    I forced myself to take a deep breath, then another, and willed my heart to slow. “I’ll do nothing,” I said. “He’s not really going to do it. He’s just trying to get attention.” With that decision, I twisted the deformation in my brain.

    The world around me skipped, displaying shattered fragments amidst frozen moments. Tom’s cat lapping at pooled blood. Sheryl finding Tom. Accusations and counter-accusations over a closed coffin. Tears, and years of recriminations.

    My gut burned, and I shuddered back to the moment. I couldn’t do nothing.

    “I’ll call the police,” I said aloud, cementing the decision in my brain. “Give them his address. Tell them what he told me. I’m going to do that next.”

    The decision changed my future, but weakened my precognition. The new future wasn’t strong yet and hadn’t yet solidified. Colors streaked the new images, and sounds skewed from lips like a badly dubbed film. Blood on Tom’s neck. A police officer tumbling down the apartment stairs, Tom’s knife buried in his arm. The stench of urine and blood. Parallel metal bars and scored, battered Plexiglas between Tom and I, new gaps in Tom’s bared teeth as he spit at me.

    Prison would be better than a coffin, maybe. But Tom wouldn’t be my friend any more. Precognition had already stolen my family, my children, and too many friends. I couldn’t stand losing another.

    “I’ll go down there myself,” I said. “See him in person. It’s only thirty minutes to his apartment. Talk to him where he can’t hang up on me.”

    Yet another change to my future distorted the new visions to the jagged edge of uselessness. I saw shards of Tom bleeding and my fingers fumbling at a phone. Tom’s voice was unintelligible, but the anger was unmistakable. A glimpse of him at a party, the years turning his hair white; he laughed, then he saw me, and he turned away. The sudden set of his jaw reminded me of my wife when we left the divorce hearing.

    The bench wobbled beneath me, the stench of bus exhaust more bitter. Tom would never forgive anyone for finding him like that. I’d save his life, but I’d lose him.

    “I’ll call him back,” I said. “I’ll talk him out of this.” I’d never viewed four futures in quick succession. My precognition showed only an unintelligible jumble that hurt my ears and eyes and left the taste of hot copper in the back of my mouth. I didn’t think that calling Tom would delay him more than a few minutes, though.

    I licked my lips and made a call.

    “Jimmy? Listen, I just heard from Tom. He’s really really upset. I’m worried about him. You live right by him, don’t you? Could you go and check on him?”

    Jimmy would find Tom, and call for help. Tom would be there for me. Tom would never forgive Jimmy for finding him, but I’d be there to console Jimmy. My friends are important. I have to keep them.

    If you enjoy this, you might look at my other free story, my available short stories or my fiction collection. If you don’t enjoy this, that’s OK. If you now feel like crossing the street when I approach, that’s fine too.

    Splitting blog?

    I usually post two different sorts of items here: tech articles, and publishing articles. Would you lot prefer I did two separate blogs? I would probably still feed both to third parties such as Twitter and Facebook, but it appears that most of my readers use RSS.

    If nobody cares, I’ll leave things as they are.

    Floating business ideas past my readers

    As I beaver away on the new Absolute OpenBSD book, I’m pondering options for what to do afterwards. Part of that pondering concerns the business aspect of publishing. And I want your opinion.

    This blog post is about tech books — or, more generally, “highly researched non-entertainment nonfiction,” a category which includes but is not limited to technology books. I’m explicitly excluding fiction and entertainment nonfiction. I’m discussing books meant to help the reader make more money, or at least keep their job.

    I’ve wanted to write about certain technologies for years, but there aren’t enough buyers to support a traditional publishing run. They’re topics that would appeal to a majority of my blog readers, but a few hundred readers just can’t support a traditionally-published book. If I self-publish on such topics, I would get more money per reader. This could make special-interest books sufficiently profitable for me to invest a year writing them.

    My goal is to make “enough” money so that I feel it’s worth spending my evenings and weekends writing a book. The exact value of “enough” varies with the topic, how hard the book is to write and research, how much I have to spend to write the book, who I have to work with to write the book, and what exactly I gave up in favor of writing the book. (Yes, I’d like to make great big steaming HEAPS of money. But that’s not realistic.) To achieve this, I must set the price of a book such that the reader feels he’s getting fair value, but still puts “enough” money in my pocket.

    The problem comes in the payments I receive on the book.

    You’ve probably heard that Amazon pays 70% royalties on self-published ebooks. That’s not quite accurate. It pays 70% royalties on self-published ebooks with a retail price of $9.99 or less. Barnes & Noble has a similar policy (look under Pricing and Payment Terms). Smashwords has a more complex royalty system, because they feed multiple ebook vendors. Royalties on books bought directly from Smashwords are about 85%, but royalties through various platforms that they feed pay varying percentages up to certain ceilings. For example, Kobo pays 60% up to $12.99, and 38% above that.

    Physical book pricing is simpler. I get a certain amount for sales through Amazon, and a lower amount for sales through third parties such as Barnes & Noble or indie bookstores. Those royalties don’t have artificial ceilings.

    I have no problem giving an ebook retailer their fair cut for delivery. I don’t wish to waste my time building and maintaining an ebook store when I could be writing. But the royalty scheme used by the large ebook retailers is clearly aimed at novels.

    Companies like Amazon and B&N want self-published novels to be priced under $10. But there’s a definite difference between a 100,000-word novel with a potential audience of millions and a 300,000-word technology book with a potential audience of hundreds.

    I cannot afford to spend a year writing a book with 500 expected buyers and sell it for $9.99. The income is not “enough.” Once I raise the price over $9.99, however, my royalty is halved. To raise my income a penny, I must increase the ebook price to over $20.

    Unfair? Probably. Unnecessary? I’d say so. But that’s the retailer’s business decision, and I cannot change it, waste my time griping about it, or go on a long rant about how companies X, Y, and Z are destroying all that is good and wholesome in the world. (They aren’t, by the way. But that’s a separate blog post.)

    So, for the sake of a purely hypothetical business decision, let me make up some numbers and facts. The pedantic will note that I’m rounding everything to the nearest dollar, but I’m already making up my own numbers, so who cares?

    Assume I want to write a hefty book about a hypothetical project, MaguffinBSD. This project will take a year, expenses are minimal, and I have friends, allies, and supporters in the community. I decide that $14,000 gross is “enough”. My research indicates that maybe 500 people will buy the book. (How do I get that number? The community is about 1/10th the size of FreeBSD’s, and Absolute FreeBSD sold about 5000 copies in the first three years, with a dwindling long tail thereafter.) Let’s also assume that the book is up to my usual standards; it’s readable, mostly free of really blatant errors, and so on.

    500 customers to raise $14,000 means that I must extract $28 from each buyer.

    Option 1: I set the ebook price at $80, and sell it at that price across all platforms. Per various terms of service, the ebook must be priced at least 20% cheaper than the physical book retail price, so the print book is $100. My profit on the physical book is much higher, but sales are much lower.

    Option 2: I write four smaller books: “MaguffinBSD, vol 1: Base Configuration,” “vol. 2, services,” “vol. 3, ongoing support,” and “vol 4: stupid MaguffinBSD Tricks.” Each of these books is available at all ebook retailers. I price each at $9.99.

    A “MaguffinBSD, vols 1-4” is available as a print book, with a consolidated index and Table of Contents.

    The version that appears in print is available as an ebook via Smashwords, and only Smashwords. It would not go to the other ebook retailers fed by Smashwords. Where you would pay $39.96 to buy each individual volume, I could sell the compendium for $32.

    People who want individual volumes have the option to get them. People who want the compendium can get it in any desired format.

    Option 3: Kickstarter. I include this because someone’s going to suggest it. I don’t like kickstarting books. Yes, some people do it, but publishing is a business. If I ever hope to make a living at writing, I need to treat it as a business. You can apply this same reasoning to asking for donations.

    Model 2 increases my expenses and production time. I must prepare one book five times, in three different formats. But I might pick up some extra readers who are only interested in one or two volumes of the set, so I’ll consider that a wash.

    But my gut reaction to model 1 is: oh dear God, NO.

    So, my question to you lot is: which model would you accept more? Which would be more offensive? Or should I give up on writing specialty tech books and start writing about Windows, Apple, and Linux?

    New review of “SSH Mastery”

    Samiuela LV Taufa was kind enough to write a review of SSH Mastery. Thank you, sir!

    For those who are wondering why I haven’t posted much lately: I’m beavering away at the new Absolute OpenBSD, getting ready for a summer writing workshop with Kris Rusch, trying to get an article together for BSD Magazine, and when my brain is too tired to put words together, assembling a print version of Vicious Redemption.

    So yes, I’m working. You just can’t see any results yet.

    Truth versus Art

    There’s been a slow-burning furor over dishonesty in “creative nonfiction,” most recently in this Fact vs. Artistic License in Creative Nonfiction post. Now and then someone accuses me of making stuff up in my books. For the record, here’s the truth.

    I lie. I make stuff up all the time. But not technical stuff.

    One technique I use in each tech book is to create a narrator. The narrator is not me. I don’t actually blackmail coworkers, as the narrator of Network Flow Analysis recommends. The narrator’s role is to bring life to the material, point out possibilities that are difficult to expose in pure technical text, and try to jolt the reader into paying attention.

    I don’t create the narrators beforehand. They evolve from the material. The narrator of AO2e is worryingly like forensic blood spatter analyst Dexter Morgan. I’m trying to change that, but he’s fighting back.

    Of course, some things are true. The afterword for Absolute OpenBSD 2nd ed. is the true story of a really bad night. But I don’t have enough of those stories to color a book.

    This technique works. It helps the reader pay attention. Some people even find reading my books enjoyable (for example, there’s this review that made me giggle madly). There are readers who hate my books for exactly this reason. But I’m not going to change my writing style to chase a readership.

    If I’m giving instructions on how to fdisk and disklabel a hard drive, the information is as correct as I can make it. Facts are inviolate.

    If it’s more personal, it might be true. It might be fictional. I am a writer, and am not to be trusted.

    So don’t try to call me out on this. I know. I don’t care.

    Death of a Web Server

    My first day at BSDCan, my Web sites died. Hard drive failure. The latest backups are defective. I think I’ve recovered the blog, but some links have changed, dang it. I’ll have to learn more about mod_rewrite to fix them. Web site is next. RSS readers will see some repeats, sorry.

    Other than that, BSDCan was awesome. As usual. In fact, it was just awesome as expected. So it was kind of routine. But still awesome.