BSDCan 2024 Reorganization

Dan Langille has spent a good part of the last twenty years on BSDCan. We’ve had other BSD conferences in the Western Hemisphere, but BSDCan is the most consistent. Covid interrupted it, but only because Dan coordinated with EuroBSDCon to have a single online conference in 2021.

This is a lot of work. Dan’s life has changed.

Dan is stepping back from organizing BSDCan. I am taking over coordinating 2024.

Note I did not say “running.” Running an international conference is a job best accomplished by a team. A large team. Dan set up BSDCan 2023 with himself and Adam Thompson, and ran it with assistance from Dru Lavigne and Warren Block in registration, and Patrick McEvoy and Andrew Fengler in streaming. I am not nearly that tough.

Instead, we have assembled a team of seventeen people to be the BSDCan 2024 Operations Team. Dan will become our source of knowledge, telling us who to talk to at the University of Ottawa and where to reliably get T-shirts locally and which caterers are most likely to cause indigestion. I am pleased that Adam, Dru, Warren, Patrick, and Andrew all cheerfully agreed to continue in their roles. (Adam’s work of coordinating travel and accommodations for the speakers will be split among a few folks, led by Adam.)

My entire job will be to coordinate the team, help them gather resources, and mediate conflicts between them. Every person on this list is motivated to make BSDCan 2024 happen, but even the best-intentioned folks can disagree. If required I’ll make final decisions, sure, but if my decision makes people unhappy, I have no doubt that the esteemed program committee will tell me I’m being an idiot. They have final say over the con, after all.

This means I need to be staunch on not doing anything myself–although I admit, I might make a call to the Diefenbunker to see how much it would cost to host the closing social event there, and to Pili Pili Chicken for a price quote to cater it. This is mostly for my peace of mind, however. Confirming it’s too expensive will put the idea to rest.

Organizing BSDCan with two folks is a monumental achievement. I have seventeen, which I consider barely adequate for a Redundant Array of Independent Dans. We’ll need folks to handle a variety of smaller tasks, from checking video times to hauling boxes. (If you haven’t hauled a box for the con staff, have you really been to BSDCan?) I intend to make use of the bsdcan-volunteers mailing list to gather those folks. Folks on our team can ask for specific help on that list, whether it be figuring out a balky database or showing up at 7AM opening day with a roll of duct tape and a cattle prod. (“Cattle prod? Really?” Hey, I don’t know the details. Knowing the details would get in the way of me doing my job. I trust the various team members to know what they need and to ask for it.)

One thing that the BSD community has historically excelled at is passing leadership from one generation to the next. BSDCan’s operations team will follow that example. We have old hands taking the lead on parts of BSDCan, but we have at least two people covering each responsibility and at least one of them should be comparatively young.

I don’t intend to coordinate BSDCan for more than a couple years. My goal is to set up a self-perpetuating structure, make sure it runs, and walk away. Normally, I wouldn’t take on anything like this, but BSDCan is important to my people and it deserves my time and attention.

The BSDCan 2024 goal is to largely reimplement BSDCan 2023, but supported by different people. Yes, there’s room to change things. Yes, I have ideas. Many people have ideas. You probably have ideas. Our overwhelming goal is to make the conference happen. Perhaps that’s unambitious but extracting knowledge from Dan’s head, documenting it, and reimplementing it will take time and energy. I don’t want to burn anyone out. I intend to retain the location, the mask policy, the papers committee, the social event, everything, until we have BSDCan down cold.

Dan has done well. He’s earned a break.

We are drafting him to run the auction, however.

BSDCan 2023 Tutorial: OpenBSD Storage Management

I’ll be teaching a four-hour tutorial on OpenBSD storage management at BSDCan 2023. As you might imagine, it’s based on OpenBSD Mastery: Filesystems.

I am pleased to see BSDCan returning to meatspace. Yes, the pandemic is ongoing, and I don’t blame folks who decide not to attend. The main reason I chose to attend is that the concom (which I’m a member of) has chosen to enforce a stringent mask policy. Yes, I know you have the right to not wear a mask in public. BSDCan is a private event for a community, however, and communities have a responsibility to protect their weakest members. (People who think that your rights outweigh your responsibilities, I will delete your comments.)

I hope to see many of you at the con, if not my tutorial. It will be good to see many old friends. Well, at least their eyes. Faces in one of the many fine outdoor dining establishments Byward Market offers.

Jan/Feb 2023 Column Out in the FreeBSD Journal

Somehow, I’ve written 28 “We Get Letters” columns for the FreeBSD Journal. The latest is out. I’m amazed that they haven’t given me the boot yet, especially as I’m attempting to channel Michael Bywater’s brilliant “Bargepole” column from the unforgivably murdered weekly “Punch.” At best I’ve achieved a tenth of Bargepole’s vitriolic geezerliciousness, but seeing as Bargepole contained enough vitriol to kill a beluga whale I expected them to ban me years ago.

You can get all the columns for free by trawling through the old issues, but you could also grab the collection and save yourself three years of trouble.

“OpenBSD Mastery: Filesystems” Print Status

I got the print proofs of OpenBSD Mastery: Filesystems yesterday, and they came out lovely.

I’ve gone through them and fixed some less-than-optimal layout decisions. You never know what a print book will look like until you hold it in your hands. Several folks read the ebook already and sent some corrections that I, the tech editors, and the copyeditor missed, so I fixed them. The ebook has been updated to match. The printer now has the print files.

So, when will the print book be in stores?

That depends on the printer. I expect they’ll approve the print in the next few days, unless they pull another Orcibus and sit on it for a month. After a week or so, though, I have nothing better to do than give them pain every. single. day.

I will release the print book to the public when I can. I’ll be ordering the pre-orders directly from the printer at the same time.

Fediverse Servers, plus mac_portacl on FreeBSD

One of my business mantras is “control your platform.” If you build your business around a site like Facebook, they can de-prioritize you and disappear you. Twitter’s implosion served as a fierce reminder of that, so I’m blogging more here.

Before Twitter’s implosion, the Fediverse (Mastodon, PixelFed, and all the other ActivityPub-powered systems) drove just as much traffic to my site as Twitter. Other social networking sites are negligible. If I want to follow my business mantra, I must run my own Fediverse server. I tested three options: Mastodon, pleroma, and GoToSocial.

Mastodon is huge, clunky, and handles like a tank made out of chicken wire, tar, and lobsters. I spoke with a few Mastodon operators, and none of them recommended it.

Pleroma? I followed the instructions. They didn’t work. I went looking into support, but I discovered that Pleroma seems to be the server of choice for TERFs, racists, and related jerks. Their recommended servers for new users are all on my personal blocklist. I don’t care to help those folks debug their instructions.

GoToSocial was a joy. Except it’s not only in development, it’s in alpha. They are very clear about this. The features that exist are beautifully done, but certain features I find critical are incomplete.

I have decided to wait to deploy a production fediverse server until GoToSocial enters beta.

For incomplete software, though, GoToSocial is surprisingly complete. It has its own web server and Let’s Encrypt implementation. If it can bind to ports 80 and 443, you don’t need a web server or ACME agent. The catch is, gotosocial(8) runs as an unprivileged user. It can’t bind to privileged ports.

Enter mac_portacl(4).

In the BSD tradition, the man page details everything you can do with this Mandatory Access Control kernel module, but in short it lets you permit particular users or group to bind to privileged network ports. I don’t care for mac_portacl in production, as the rules are hard to read when you’re debugging. If you want me to use an access control program, the output better be no harder to read than pfctl -sr. But here’s how you do it.

Enable the module in /boot/loader.conf.

mac_portacl_load="YES"

You can now write port ACL rules. Each rule has four parts:

uid or group : numerical identifier : tcp or udp : port number

The gotosocial user has uid 209. I want uid 209 to be able to bind to TCP ports 80 and 443, so I need these rules.

uid:209:tcp:80
uid:209:tcp:443

Set the access control rules in /etc/sysctl.conf.

net.inet.ip.portrange.reservedhigh=0
security.mac.portacl.rules=uid:209:tcp:443,uid:209:tcp:80

The first sysctl disables the traditional “reserved port” behavior and allows unprivileged programs to bind to ports below 1024.

The second sysctl installs our rules in the kernel. When you write to this sysctl you must include all rules you want active, separated by commas.

Would I use this in production? If the software has a solid security track record and is designed to be directly exposed to the Internet, sure. If you’re running a web server, some program has to listen on port 80. GoToSocial is brand new, though, and I’d like to see a bit of a track record before I completely trusted it.

When GoToSocial enters beta next year and I deploy it for real, I’ll put an nginx or httpd in front of it so I can filter when needed.

Are there other options other than Mastodon, pleroma, and GoToSocial? Sure. But I’m out of time, and really need to make some words this week.

Two pieces by me in this month’s FreeBSD Journal

Yes, I’m trying to use the blog more, rather than dumping everything to multiple social media outlets. Yes, this is in part in response to Comic Book Supervillain purchasing Twitter and kneecapping the moderation team. If you want me on social media, I’m on the fediverse as @mwlucas@bsd.network.

Anyway.

The latest issue of the FreeBSD Journal has two articles by me: one on PAM tips & tricks, and the other my regular “We Get Letters” “advice” column. With any luck, the Journal’s editorial board will use these articles as grounds for reconsidering their “we’ll publish anything Lucas sends us” policy.

If you find the Letters column amusing, I’ve collected the first three years of that column in Letters to ed(1).

upgrading PHP 7.4 to PHP 8 on FreeBSD

What, a technical post? It happens. Rarely. Usually, I’m focused on the tech that goes into a book, but sometimes the real world intervenes.

Like PHP. PHP is very much the real world. My site has been running PHP 7.4 for a while, which goes end of life on 28 November. I put this off as long as possible, but it’s time to update.

I run my e-bookstore on Woocommerce, which is built on WordPress, which is built on PHP. What started as a silly experiment has become the center of my business. I need to minimize downtime, which means I must check everything before upgrading. It’s PHP, which means it’s a maze of twisty little modules that all look alike. PHP has this annoying habit of adding, removing, splitting, and changing modules. Running PHP applications on FreeBSD is all about finding the module your application needs, so I want to identify all possible problems before changing.

First, let’s see what packages need upgrading.

# pkg info -x php
mod_php74-7.4.32_1
php74-7.4.32
php74-ctype-7.4.32
php74-curl-7.4.32
php74-dom-7.4.32
php74-exif-7.4.32
php74-fileinfo-7.4.32
php74-filter-7.4.32
php74-gd-7.4.32
php74-iconv-7.4.32
php74-intl-7.4.32
php74-json-7.4.32
php74-mbstring-7.4.32
php74-mysqli-7.4.32
php74-openssl-7.4.32
php74-pcntl-7.4.32
php74-pdo-7.4.32
php74-pdo_mysql-7.4.32
php74-pecl-imagick-im7-3.5.1_1
php74-phar-7.4.32
php74-posix-7.4.32
php74-session-7.4.32
php74-simplexml-7.4.32
php74-soap-7.4.32
php74-tokenizer-7.4.32
php74-xml-7.4.32
php74-xmlreader-7.4.32
php74-xmlrpc-7.4.32
php74-xmlwriter-7.4.32
php74-zip-7.4.32_1
php74-zlib-7.4.32

31 packages. Software like Tiny Tiny RSS and WordPress depend on PHP, but if the underlying PHP software has all the necessary libraries then they should just work. Should. But PHP modules sometimes disappear, get replaced, or get renamed. I want a list of all the modules I need before running any commands. So, what would the PHP 8.0 version of these packages be named? I have to iterate through sed a couple times to trim out excess version information and wind up with this.

# pkg info -x php | sed s/74/80/g | sed s/-7.4.32//g | sed s/_1//g

mod_php80
php80
php80-ctype
php80-curl
php80-dom
php80-exif
php80-fileinfo
php80-filter
...

Those look sensible. Now check to see if the packages exist.

I could automate this by checking the exit code of each command, but the list is short enough that I can process it by hand. I run one package search at a time, letting xargs prompt me for each one so I can eyeball the results.

# pkg info -x php | sed s/74/80/g | sed s/-7.4.32//g | sed s/_1//g | xargs -L1 -p pkg search
pkg search mod_php80?…y
mod_php80-8.0.25 PHP Scripting Language
pkg search php80?…y

This particular search will spew a couple hundred lines of output, but I’m confident the base PHP 8.0 package is in there.

...
php80-intl-8.0.25 The intl shared extension for php
pkg search php80-json?...y
pkg search php80-mbstring?...

Ooops! Pay attention here. There is no package for PHP 8.0’s JSON module! Make a note of that.

At the end, I have problems with three packages: php80-json, php80-openssl, and php80-xmlrpc. Freshports tells me that the JSON and OpenSSL modules were added into the default PHP 8.0 package, so I can cross those off my list.

The XML-RPC module is another tale. PHP 8.0 no longer has an XML module. Fortunately, that same bug lists a replacement pecl-xmlrpc. There’s a related php80-pecl-xmlrpc module.

I have a list of modules to install. For a last check, I’ll look for anything that depends on PHP 7.4.

# pkg info -dx php74
The list looks different, but contains the same modules. I’m as prepared as I can be.

One last check. Make a list of the packages to install. Eyeball it to make sure it looks right.

# pkg info -x php | sed s/74/80/g | sed s/-7.4.32//g | sed s/_1//g > php8.pkg

Create a boot environment, and do a dry run. If I remove all packages with PHP in their name, what will get pulled? Using -n tells me what the command would do, but doesn’t actually change anything.

# bectl create 12.3-p7-lastbeforePHP
# pkg remove -nx php74

That list looks sensible. Now remove the packages, and install everything on our list.

# pkg remove -x php74
# cat php8.pkg | xargs -L1 -p pkg install -y

The -p argument to xargs prompts me for confirmation, so I can use -y on the pkg command. The install fails on the nonexistent JSON, OpenSSL, and XMLRPC modules, but that’s expected.

At the end, I manually install php80-pecl-xmlrpc.

Reboot.

Test, test, test. Run a test purchase. It works.

Everything looks okay? I guess I can turn it over to the Crowdsourced Monitoring System, aka “y’all,” and go make some paying words.

“OpenBSD Mastery: Filesystems” draft done!

After far too long, I have finished a first draft of OpenBSD Mastery: Filesystems. Sponsorships are now closed.

I’m asking tech reviewers to get any comments to me by 15 October 2022. That’s four weeks. It might seem tight, but experience shows that people either get their comments to me immediately, or wait until the last possible weekend. I’m not complaining–I do exactly the same thing. Please return any comments either a) in plain text, with enough context that I can find them when page numbers change, or b) as annotations directly on the PDF.

My tech reviewers are now in their third decade of winning the prize for “most likely to use many different PDF readers.” A file that works for one won’t work for another. I work around this by distributing three PDFs of the manuscript, each identical in contact but prepared differently. Everyone should be able to find one that works for them.

If you’re interested in doing a tech review, please drop me an email (mwl at mwl dot io) saying who you are, why you would make you a good reviewer, and that you won’t share the manuscript. (Piracy is bad, but having my name on an unreviewed and thus certainly incorrect document is horrifying). I’ll ignore responses that can’t follow those instructions, because whenever I don’t I get difficult-to-decipher feedback. (I have previously received PostScript diffs, and… no. Just no.)

I’ll be turning my attention to the Prohibition Orcs copyedits next. Then it’s back to the Epic Giant Fiction Project, and another tech book, title TBA.

“OpenBSD Mastery: Filesystems” Status Report

I just finished the ‘non-native filesystems’ part of “OpenBSD Mastery: Filesystems.” I wouldn’t say I’ve finished the hard part, but I have finished the “intertwined to an unholy degree” part.

In the beginning, Berkely released Unix. This made a lot of vendors very angry and has widely been regarded as a bad move.

Why have I spent months on five chapters? Because everything in the core storage system of any Unix is intertwined to a nearly unholy degree. To understand filesystems you must understand partitioning, but to understand why Unix uses partitions as it does you need to understand filesystems. I have to meticulously disentangle facts so that I can start explanations at the bottom of the storage stack, but add in enough higher-level details exactly when you need them so you can make sense of why the bottom layers work as they do.

Otherwise, you’d look at computers and think “Wow, this whole thing is stupid.” Don’t get me wrong, the whole thing IS stupid, but it’s your job to understand the stupidity and I don’t need to be rubbing your nose in it.

Have I written on these before?

Yes, many times.

Does that make them easier to write?

BWAHAHAHAHA. No.

Can I use the earlier edition of Absolute OpenBSD to guide me?

Sure, except that the book is ten years old and every detail within is suspect and must be triple-checked against the current state of the software and oh by the way that book doesn’t even mention GPT or FUSE so burn it all down. AO2e is a checklist of things that will annoy me.

The good news is, the sections that remain are fairly tidy. They’re not standalone, but they are less incestuously intertwined with other topics.

  • NFS
  • iSCSI
  • softraid
  • encrypted storage

    The first two are mostly standalone, and are thus easier to write. Also, as an author I am highly grateful that OpenBSD does not support NFSv4.

    I’m going to push hard to get this done in the next few weeks. Which brings me to:

    Once that happens, sponsorships will close. If you want your name in the book, act now.