BSDCan 2023 Tutorial: OpenBSD Storage Management

I’ll be teaching a four-hour tutorial on OpenBSD storage management at BSDCan 2023. As you might imagine, it’s based on OpenBSD Mastery: Filesystems.

I am pleased to see BSDCan returning to meatspace. Yes, the pandemic is ongoing, and I don’t blame folks who decide not to attend. The main reason I chose to attend is that the concom (which I’m a member of) has chosen to enforce a stringent mask policy. Yes, I know you have the right to not wear a mask in public. BSDCan is a private event for a community, however, and communities have a responsibility to protect their weakest members. (People who think that your rights outweigh your responsibilities, I will delete your comments.)

I hope to see many of you at the con, if not my tutorial. It will be good to see many old friends. Well, at least their eyes. Faces in one of the many fine outdoor dining establishments Byward Market offers.

Jan/Feb 2023 Column Out in the FreeBSD Journal

Somehow, I’ve written 28 “We Get Letters” columns for the FreeBSD Journal. The latest is out. I’m amazed that they haven’t given me the boot yet, especially as I’m attempting to channel Michael Bywater’s brilliant “Bargepole” column from the unforgivably murdered weekly “Punch.” At best I’ve achieved a tenth of Bargepole’s vitriolic geezerliciousness, but seeing as Bargepole contained enough vitriol to kill a beluga whale I expected them to ban me years ago.

You can get all the columns for free by trawling through the old issues, but you could also grab the collection and save yourself three years of trouble.

“OpenBSD Mastery: Filesystems” Print Status

I got the print proofs of OpenBSD Mastery: Filesystems yesterday, and they came out lovely.

I’ve gone through them and fixed some less-than-optimal layout decisions. You never know what a print book will look like until you hold it in your hands. Several folks read the ebook already and sent some corrections that I, the tech editors, and the copyeditor missed, so I fixed them. The ebook has been updated to match. The printer now has the print files.

So, when will the print book be in stores?

That depends on the printer. I expect they’ll approve the print in the next few days, unless they pull another Orcibus and sit on it for a month. After a week or so, though, I have nothing better to do than give them pain every. single. day.

I will release the print book to the public when I can. I’ll be ordering the pre-orders directly from the printer at the same time.

Fediverse Servers, plus mac_portacl on FreeBSD

One of my business mantras is “control your platform.” If you build your business around a site like Facebook, they can de-prioritize you and disappear you. Twitter’s implosion served as a fierce reminder of that, so I’m blogging more here.

Before Twitter’s implosion, the Fediverse (Mastodon, PixelFed, and all the other ActivityPub-powered systems) drove just as much traffic to my site as Twitter. Other social networking sites are negligible. If I want to follow my business mantra, I must run my own Fediverse server. I tested three options: Mastodon, pleroma, and GoToSocial.

Mastodon is huge, clunky, and handles like a tank made out of chicken wire, tar, and lobsters. I spoke with a few Mastodon operators, and none of them recommended it.

Pleroma? I followed the instructions. They didn’t work. I went looking into support, but I discovered that Pleroma seems to be the server of choice for TERFs, racists, and related jerks. Their recommended servers for new users are all on my personal blocklist. I don’t care to help those folks debug their instructions.

GoToSocial was a joy. Except it’s not only in development, it’s in alpha. They are very clear about this. The features that exist are beautifully done, but certain features I find critical are incomplete.

I have decided to wait to deploy a production fediverse server until GoToSocial enters beta.

For incomplete software, though, GoToSocial is surprisingly complete. It has its own web server and Let’s Encrypt implementation. If it can bind to ports 80 and 443, you don’t need a web server or ACME agent. The catch is, gotosocial(8) runs as an unprivileged user. It can’t bind to privileged ports.

Enter mac_portacl(4).

In the BSD tradition, the man page details everything you can do with this Mandatory Access Control kernel module, but in short it lets you permit particular users or group to bind to privileged network ports. I don’t care for mac_portacl in production, as the rules are hard to read when you’re debugging. If you want me to use an access control program, the output better be no harder to read than pfctl -sr. But here’s how you do it.

Enable the module in /boot/loader.conf.


You can now write port ACL rules. Each rule has four parts:

uid or group : numerical identifier : tcp or udp : port number

The gotosocial user has uid 209. I want uid 209 to be able to bind to TCP ports 80 and 443, so I need these rules.


Set the access control rules in /etc/sysctl.conf.


The first sysctl disables the traditional “reserved port” behavior and allows unprivileged programs to bind to ports below 1024.

The second sysctl installs our rules in the kernel. When you write to this sysctl you must include all rules you want active, separated by commas.

Would I use this in production? If the software has a solid security track record and is designed to be directly exposed to the Internet, sure. If you’re running a web server, some program has to listen on port 80. GoToSocial is brand new, though, and I’d like to see a bit of a track record before I completely trusted it.

When GoToSocial enters beta next year and I deploy it for real, I’ll put an nginx or httpd in front of it so I can filter when needed.

Are there other options other than Mastodon, pleroma, and GoToSocial? Sure. But I’m out of time, and really need to make some words this week.

Two pieces by me in this month’s FreeBSD Journal

Yes, I’m trying to use the blog more, rather than dumping everything to multiple social media outlets. Yes, this is in part in response to Comic Book Supervillain purchasing Twitter and kneecapping the moderation team. If you want me on social media, I’m on the fediverse as


The latest issue of the FreeBSD Journal has two articles by me: one on PAM tips & tricks, and the other my regular “We Get Letters” “advice” column. With any luck, the Journal’s editorial board will use these articles as grounds for reconsidering their “we’ll publish anything Lucas sends us” policy.

If you find the Letters column amusing, I’ve collected the first three years of that column in Letters to ed(1).

upgrading PHP 7.4 to PHP 8 on FreeBSD

What, a technical post? It happens. Rarely. Usually, I’m focused on the tech that goes into a book, but sometimes the real world intervenes.

Like PHP. PHP is very much the real world. My site has been running PHP 7.4 for a while, which goes end of life on 28 November. I put this off as long as possible, but it’s time to update.

I run my e-bookstore on Woocommerce, which is built on WordPress, which is built on PHP. What started as a silly experiment has become the center of my business. I need to minimize downtime, which means I must check everything before upgrading. It’s PHP, which means it’s a maze of twisty little modules that all look alike. PHP has this annoying habit of adding, removing, splitting, and changing modules. Running PHP applications on FreeBSD is all about finding the module your application needs, so I want to identify all possible problems before changing.

First, let’s see what packages need upgrading.

# pkg info -x php

31 packages. Software like Tiny Tiny RSS and WordPress depend on PHP, but if the underlying PHP software has all the necessary libraries then they should just work. Should. But PHP modules sometimes disappear, get replaced, or get renamed. I want a list of all the modules I need before running any commands. So, what would the PHP 8.0 version of these packages be named? I have to iterate through sed a couple times to trim out excess version information and wind up with this.

# pkg info -x php | sed s/74/80/g | sed s/-7.4.32//g | sed s/_1//g


Those look sensible. Now check to see if the packages exist.

I could automate this by checking the exit code of each command, but the list is short enough that I can process it by hand. I run one package search at a time, letting xargs prompt me for each one so I can eyeball the results.

# pkg info -x php | sed s/74/80/g | sed s/-7.4.32//g | sed s/_1//g | xargs -L1 -p pkg search
pkg search mod_php80?…y
mod_php80-8.0.25 PHP Scripting Language
pkg search php80?…y

This particular search will spew a couple hundred lines of output, but I’m confident the base PHP 8.0 package is in there.

php80-intl-8.0.25 The intl shared extension for php
pkg search php80-json?...y
pkg search php80-mbstring?...

Ooops! Pay attention here. There is no package for PHP 8.0’s JSON module! Make a note of that.

At the end, I have problems with three packages: php80-json, php80-openssl, and php80-xmlrpc. Freshports tells me that the JSON and OpenSSL modules were added into the default PHP 8.0 package, so I can cross those off my list.

The XML-RPC module is another tale. PHP 8.0 no longer has an XML module. Fortunately, that same bug lists a replacement pecl-xmlrpc. There’s a related php80-pecl-xmlrpc module.

I have a list of modules to install. For a last check, I’ll look for anything that depends on PHP 7.4.

# pkg info -dx php74
The list looks different, but contains the same modules. I’m as prepared as I can be.

One last check. Make a list of the packages to install. Eyeball it to make sure it looks right.

# pkg info -x php | sed s/74/80/g | sed s/-7.4.32//g | sed s/_1//g > php8.pkg

Create a boot environment, and do a dry run. If I remove all packages with PHP in their name, what will get pulled? Using -n tells me what the command would do, but doesn’t actually change anything.

# bectl create 12.3-p7-lastbeforePHP
# pkg remove -nx php74

That list looks sensible. Now remove the packages, and install everything on our list.

# pkg remove -x php74
# cat php8.pkg | xargs -L1 -p pkg install -y

The -p argument to xargs prompts me for confirmation, so I can use -y on the pkg command. The install fails on the nonexistent JSON, OpenSSL, and XMLRPC modules, but that’s expected.

At the end, I manually install php80-pecl-xmlrpc.


Test, test, test. Run a test purchase. It works.

Everything looks okay? I guess I can turn it over to the Crowdsourced Monitoring System, aka “y’all,” and go make some paying words.

“OpenBSD Mastery: Filesystems” draft done!

After far too long, I have finished a first draft of OpenBSD Mastery: Filesystems. Sponsorships are now closed.

I’m asking tech reviewers to get any comments to me by 15 October 2022. That’s four weeks. It might seem tight, but experience shows that people either get their comments to me immediately, or wait until the last possible weekend. I’m not complaining–I do exactly the same thing. Please return any comments either a) in plain text, with enough context that I can find them when page numbers change, or b) as annotations directly on the PDF.

My tech reviewers are now in their third decade of winning the prize for “most likely to use many different PDF readers.” A file that works for one won’t work for another. I work around this by distributing three PDFs of the manuscript, each identical in contact but prepared differently. Everyone should be able to find one that works for them.

If you’re interested in doing a tech review, please drop me an email (mwl at mwl dot io) saying who you are, why you would make you a good reviewer, and that you won’t share the manuscript. (Piracy is bad, but having my name on an unreviewed and thus certainly incorrect document is horrifying). I’ll ignore responses that can’t follow those instructions, because whenever I don’t I get difficult-to-decipher feedback. (I have previously received PostScript diffs, and… no. Just no.)

I’ll be turning my attention to the Prohibition Orcs copyedits next. Then it’s back to the Epic Giant Fiction Project, and another tech book, title TBA.

“OpenBSD Mastery: Filesystems” Status Report

I just finished the ‘non-native filesystems’ part of “OpenBSD Mastery: Filesystems.” I wouldn’t say I’ve finished the hard part, but I have finished the “intertwined to an unholy degree” part.

In the beginning, Berkely released Unix. This made a lot of vendors very angry and has widely been regarded as a bad move.

Why have I spent months on five chapters? Because everything in the core storage system of any Unix is intertwined to a nearly unholy degree. To understand filesystems you must understand partitioning, but to understand why Unix uses partitions as it does you need to understand filesystems. I have to meticulously disentangle facts so that I can start explanations at the bottom of the storage stack, but add in enough higher-level details exactly when you need them so you can make sense of why the bottom layers work as they do.

Otherwise, you’d look at computers and think “Wow, this whole thing is stupid.” Don’t get me wrong, the whole thing IS stupid, but it’s your job to understand the stupidity and I don’t need to be rubbing your nose in it.

Have I written on these before?

Yes, many times.

Does that make them easier to write?


Can I use the earlier edition of Absolute OpenBSD to guide me?

Sure, except that the book is ten years old and every detail within is suspect and must be triple-checked against the current state of the software and oh by the way that book doesn’t even mention GPT or FUSE so burn it all down. AO2e is a checklist of things that will annoy me.

The good news is, the sections that remain are fairly tidy. They’re not standalone, but they are less incestuously intertwined with other topics.

  • NFS
  • iSCSI
  • softraid
  • encrypted storage

    The first two are mostly standalone, and are thus easier to write. Also, as an author I am highly grateful that OpenBSD does not support NFSv4.

    I’m going to push hard to get this done in the next few weeks. Which brings me to:

    Once that happens, sponsorships will close. If you want your name in the book, act now.

  • Auction Against Human Trafficking

    My wife works with Soroptimist International of Grosse Pointe to support their anti-human-trafficking efforts. They usually hold a variety of fundraisers, including a 5K race in September.

    2020 and 2021 have been less than optimal fundraising years. Running a 5K on your own is much less interesting than trying to outrace a whole bunch of folks.

    This year SIGP is raising money for the Joseph Project, a non-for-profit organization who connect human trafficking survivors with skilled pro bono legal counsel. One of the worst parts of human trafficking is that people must break the law to survive, let alone escape. Survival usually means a criminal record. I’ve written about human trafficking before, and completely support the efforts to give these folks a new chance.

    Meanwhile, here I am with a variety of stuff that’s explicitly designed to raise money in the most daft way possible, and a whole mass of readers who couldn’t be unemployed if they tried. To try to cover the gap, I’m auctioning off something that was never before available online, and never will be again.

    The Bail Bond Denied edition of FreeBSD Mastery: Jails.

    Only five copies of this edition exist. One is in my archives. One was auctioned off at BSDCan 2019, the other at Penguicon 2019. The fourth will be reserved for the next charity auction I attend in person.

    The fifth is being auctioned off right here. The members of the Grosse Pointe Soroptimists Club are signing it to express their gratitude for your generosity. I’ll sign it myself, thereby destroying their added value.

    This is your only opportunity to get this book without being present at an event I’m at. Yes, I’m opening this up to seven billion people.

    The auction rules are simple:

    Comment on this post to bid. I advise bidders to click the “notify me of new comments by email” button, so they can see when they’re outbid.

    The auction runs from now until 5 PM EDT 25 September. Most of the bidding happens on the first and last days, but I want to leave time for the Internet’s slow publicity machine to catch on. If bidding is frenzied in those last moments, I’ll leave the auction running to squeeze the most cash out of you get the best result for the Soroptimists.

    Once the auction closes, send your donation to SIGP Paypal, by following the link on and send me your receipt. I will mail you the book at my expense.

    Note that you don’t need to win a silly book to donate. You can donate out of your own generosity. I promise they’ll appreciate it. If you need someone to blame to keep your hard-boiled reputation, tell them I made you do it.