Kansas or Minneapolis Fundraiser

My country is in trouble, and I’m just a tiny rat dude making a marginal living writing about stuff very few folks care about. Not much I can do directly.

My 1 April Kickstarter required some unusual tests, so I produced a physical artifact of interest to a certain subset of sysadmins. I have that artifact. It’s a real thing. I have to make a couple tweaks and add tidbits here and there, but the thing exists.

You want proof it’s real? Here you go!

Mildly redacted to preserve the surprise. Also, this photo is legitimately a hint.

Folks have started asking what it is. I’m not saying until the Kickstarter launches.

I am, however, auctioning off the prototype for charity.

Yes, I’m asking folks to give money for a sysadminny thing sight unseen. The money doesn’t go to me, however! I want you to support a worthy cause. I have a choice of worthy causes: supporting Minnesota folks trapped by ICE, or helping trans folks in Kansas.

What do you get?

When you send me the receipt for your donation, I will mail you the thing and send you a Kickstarter preview link. I will also include a letter declaring that your direct financial support of charity grants you moral superiority. Once the final product escapes, I’ll ship you one of those. So really, the physical good will be unique for only a few weeks.

Intangibly, though? Ah, the intangible benefits! BRAGGING RIGHTS. You’ll know what the thing is before anyone else! You’ll have grounds to call me a dumbass before anyone else! People will call me a dumbass anyway, but your comments will be evidence-based and thus folks will take you seriously.

I will pay for standard Priority Mail shipping. That’ll be three days within the US, and a couple weeks overseas. (If you’re outside the US and want fast shipping, I’ll ask that you send me a few bucks for the upgrade. Sorry, but overnight to Germany or Australia ain’t cheap.)

All I ask is that you don’t ruin the surprise for folks. If you blab, I can’t do much. Sure, I’ll never do a fundraiser like this again and I’ll call you a jerk, but that’s about it. Unless you live within wedgie distance.

I am, of course, perfectly fine if you post something like “holy crap Lucas is a jerk this thing is a total ripoff like I’ve never seen before don’t you dare go to https://mwl.io/ks and follow it you’ll only encourage his next lame travesty.”

Bid by leaving a comment on this page.

The auction runs from now until 5PM EDT on 9 March 2026. If the bidding goes nuts in the last few minutes, I’ll leave it open until it settles down. There’s no sniping this auction at the last moment, as I want bids to escalate beyond all sensible limits.

The winner gets to pick a charity off of https://www.standwithminnesota.com/ and $a_Kansas_trans_charity_I’m_picking_with_the_help_of_folks_on_the_fediverse and donate their bid. Send me the receipt and I’ll send you the thing. I want you to be able to enjoy knowing the secret for as long as possible, so I’ll ship it ASAP.

Bid early! Bid often! Bid to be the first one disappointed!

112: A Special Uberblock

OpenZFS Mastery is staggering along. Here we talk about how ZFS maintains uber-integrity.

Not having dedicated special index blocks sounds great, but every data tree needs a root. ZFS stores a pointer to the filesystem root in a special uberblock. Every pool has a queue of 128 uberblocks stored at algorithmically-predictable locations. In keeping with the copy-on-write design (Chapter XXX), uberblocks are never edited. Every time a new transaction group gets written to disk, ZFS records the new root information in the next uberblock in line. When the 128th uberblock is used, ZFS loops back to the beginning. At boot, the system searches for the uberblock with the highest transaction group number and uses that to find the pool’s root. If the newest uberblock appears damaged or incomplete, ZFS falls back to the newest usable uberblock. A badly timed failure might cost you the latest transaction group’s worth of data, but the pool itself will be coherent and will not require an integrity check.

OpenZFS Mastery is still open for sponsorship.

111: Artifically Prolonged, Unnecessarily Stressful

Here’s OpenZFS Mastery on physical labeling. I have strong feelings on this.

Develop a consistent naming and numbering scheme for your storage arrays, and use it dogmatically. Many storage arrays have a standard naming scheme, often printed on the equipment. If your equipment already has numbered shelves, use that numbering. Otherwise, make simple rules like “shelf 0 is always at the top and disk 0 is always at the left.” You might use the prefix “f” for the front and “b” for the back, or whatever works for you.

Record the serial number of each drive as you install it in the array. Physically label each drive tray with the drive serial number and physical location. Use good labels that remain stuck over years of being ignored in a dry, dusty datacenter. Yes, this is tedious—but when a drive fails you must have this information. You can do this work in peace and quiet at your own pace, or you can desperately rush through it during an artificially prolonged, unnecessarily stressful outage.

OpenZFS Mastery is open for sponsorships.

110: Resorting to Extraordinary Means

Work is underway on OpenZFS Mastery.

ZFS can run on anything the operating system presents as a block device. The most common are disks. Spinning rust, SSD, NVMe? Sure. Virtual disk files stored on another filesystem? If that’s what you’ve got, ZFS will cope with the extra overhead and with the right settings can still protect your data. USB flash drives? The performance will be terrible, but maybe you don’t care. Memory disks? Uh… presumably you have a reason for wanting a robust ephemeral filesystem, but fine, I guess? We won’t delve into specific differences between physical media types, as any advice we might offer will be overtaken by reality. We will discuss how to use that storage, however.

You want specific hardware advice? Fine. Spinning rust is slow and inexpensive, SSDs cost more but are faster, and NVMe is blazing fast but pricier still and you can only fit a handful on a system without resorting to extraordinary means.

Sponsorships are open at https://sponsor.mwl.io.

January’s Jammed Sausage

This See the Sausage Being Made post goes to Patronizers in January and becomes public in February. Not a Patronizer? You could be! $12 a year gets you my latest updates, occasional free tidbits, and the completely pointless MWL Footnote Fortune File, freshly updated for the new edition of Networking for System Administrators.

I have one “job.” Yes, I write books. I get paid for that. But the thing I get a “paycheck” for is writing one blog post a month, at the beginning of the month. For Patronizers.

Here we are at January 28, and I am just getting to writing that blog post.

Absolutely zero excuses, only a procedural failure.

I like to write the blog post when something happens. What happened in the last six weeks?

Nothing.

Well, there’s The Longest Dark, the orcish solstice holiday. I took time off for that. Come January, I started to get to work and realized that my test lab was, at best, suboptimal. Many of the topics I write about can be tested and deployed on virtual machines, but I like to do filesystem books on real hardware or at least real disks. I have a host for that–it’s old, but who cares? Disks are disks, and this thing has eight SATA drives and a couple SATADOMs for the OS. Eight drives lets me test most common ZFS configurations. The host is large and noisy, but I have a basement and it has a BMC so I can kick it as needed. Creating a hard drive failure means two flights of stairs and opening a case, but I can live with that.

Mind you, I had stolen some of the hard drives for other purposes. More drives were needed. I descended into the Parts Closet. Well, I call it the Parts Closet. Other people call it a trash heap, or perhaps a Do-It-Yourself Toxic Gas Cloud Kit, Add Your Own Fire. Old computers, old printers, all precariously balanced. Laptops preinstalled with Windows XP with wobbly lids, cat5 wire crimping kits, pristine Unicomp Model M keyboards with button mice, Keyboardio Model 100s with Dvorak keycaps, a wall of small storage boxes with labels like SOEKRIS and PCCARD and SCSI TERMINATORS. The POWER OVER ETHERNET box sits right next to ETHERNET OVER POWER, which I do recommend as a convenient trick if the electrical system in your home doesn’t date from 1949 and wasn’t engineered by a PTSD-stricken pipefitter with delusions of grounding. (Fortunately cable TV is historical, so I can run Ethernet over the old coax someone paid big money to install.) Once I started digging for hard drives and discovered several expensive 250GB IDE disks, the necessity of a purge overcame my unwillingness to perform said purge.

Finding the cable crimpers reminded me that I needed to check some of the long-haul cabling in the house. The cable tester showed that I shouldn’t ask questions I don’t want answered.

Now let’s detour into backups.

I have a iX Systems miniXL running FreeNAS for local backups, and rely on Dropbox and Tarsnap for external backups. All the backups get tested monthly, mainly because I routinely screw up and must recover from backup. The miniXL also serves as my MP3 storage, because cleaning out the parts closet demands a soundtrack.

Naturally, I’m halfway through the ghastly purge when the music dies.

The miniXL’s boot SATADOM was no longer a boot device in the BIOS.

Fine. I’ve never worked with SATADOM hardware before, but how bad could it be? I ordered an inexpensive replacement.

The Supermicro power cable didn’t fit. Apparently that’s a well-known example of vendor lock-in, so I ordered a more expensive guaranteed compatible Supermicro SATADOM that was not at all compatible and immediately went back to the dealer. Fortunately, some ex-iX folks mentioned that the miniXL had space for an SSD. I had found a couple SSDs in the Parts Closet.

But then I realized that the miniXL has eight removable hard drive bays, is much quieter than my test system, and would easily fit under my standing desk (pic). Causing a hard drive failure with this host doesn’t require going to the basement and opening a case.

The critical service, music, is now being shared from my Mac desktop.

Backups? I have external USB cases for local hosts to back up to. I used the FreeNAS box because it was there. The miniXL is now running raw FreeBSD 15 and bhyve.

I now have a less unsuitable test environment and the toxic waste accident has been relocated from my home to the recycling center. Well, technically, I’ve converted one large toxic waste accident into two smaller accidents and added one of those accidents to a full scale disaster, but I’m not responsible for that disaster so I’m going to be a good citizen and ignore it.

In other news: last month I said I had completed fulfillment of the new Networking for System Administrators by packing and shipping a few hundred books. Uh… not so much. Yes, I finished packing the books. The US Postal Service hauled them off my porch. All good?

No.

More than one backer contacted me to say that their books were weirdly delayed. The USPS hauled my overseas books from Detroit to the big processing center Chicago, where most of them sat for a month. A handful ricocheted between Chicago and NYC for weeks. Several folks told me that this is unacceptable. I agree. I have no leverage nor any ability to fix this, however. My country is increasingly hostile to small business. I’m pretty sure that if I used USPS’ expensive two-day shipping, they would arrive speedily. I could use UPS or Fedex, except they’re expensive and might charge $50 to collect $3 in VAT depending on criteria so complicated as to be effectively unpredictable.

There’s also been other surprises. US Export Compliance returned one package because they require a full recipient name. A recipient like “A D Smith” is not acceptable; the shipping label must spell out Anonymous Doof. I’d expect the destination address to establish an internal protocol for demultiplexing initials, but apparently my government needs to know exactly who backs me. This has never been a problem before, but fine, rules change and there’s probably an announcement somewhere.

All I can think of to do is: warnings on my crowdfunding campaigns. Lots of warnings.

Much like I do when I warn you Patronizers that backing me is a terrible deal. January had no progress, but many tiny fixes. Yes, yes, that’s its own kind of progress but it doesn’t get words on the page. I do appreciate y’all. Thank you.

“OpenZFS Mastery” device names options

The original FreeBSD Mastery: ZFS books recommended managing disks by labels based on serial number.

  pool: vm
 state: ONLINE
config:

	NAME                   STATE     READ WRITE CKSUM
	vm                     ONLINE       0     0     0
	  gpt/WD-WCC4N0JSJDKF  ONLINE       0     0     0
	  gpt/WD-WMC4N0M8NRXM  ONLINE       0     0     0

errors: No known data errors

I do this on hosts with multiple storage devices. It’s great for reality, with copy-and-paste terminals. It’s terrible for an educational book. The brain cannot absorb this easily. I see two ways around this. There’s the method used in the original books:

  pool: compost
 state: ONLINE
  scan: none requested
config:

NAME       STATE READ WRITE CKSUM
compost    ONLINE   0     0     0
gpt/zfs0   ONLINE   0     0     0
gpt/zfs1   ONLINE   0     0     0

Easy to understand. A terrible example. Readers of the first book did this, despite the copious warnings not to.

It was suggested that I could use truncated fake serial numbers from different manufacturers.

  pool: vm
 state: ONLINE
config:

	NAME             STATE     READ WRITE CKSUM
	vm               ONLINE       0     0     0
	  gpt/WD-WCC4N0  ONLINE       0     0     0
	  gpt/SEA-4N0M8  ONLINE       0     0     0
	  gpt/TOSH-9262  ONLINE       0     0     0
	  gpt/NCC-1701A  ONLINE       0     0     0

errors: No known data errors

This is obviously fake. It also obviously represents serial numbers.

So, the question for my nonfiction readers is: how does the latter example stick in your brain? Is it not only readable but absorbable?

Paypal re-enabled on my store

Seems that during a round of troubleshooting the Tilted Windmill Press store, I temporarily disabled Paypal. Turning off features until you figure out what’s broken is standard troubleshooting.

Apparently I skipped the part where I turn features back on. Oops. Paypal is back. Sorry, folks.

Nobody complained, which makes me curious. Does anybody have reason to need Paypal rather than a credit card? I want to offer all reasonable payment options, but I don’t want to carry unnecessary stuff.

109: We Won’t Live Long Enough

Yes, I’ve been missing for a few weeks. Caught a bad case of Life. Here’s some OpenZFS Mastery.

ZFS advocates claim that ZFS is immune to these arbitrary limits, but that’s not quite true. ZFS store most of its values in 128-bit variables. One directory can have 248 files, of up to 16 exabytes each. A single pool can be up to 256 zettabytes, or 278 bytes. A storage pool can contain up to 264 devices, and a single host can have up to 264 storage pools. File and directory names can be up to 1023 ASCII characters.

The good news is, we will not live long enough to hit these limits. The bad news is, today’s sysadmins have all the expertise in migrating between filesystems. When technology hits ZFS’ limits, those poor people won’t be accustomed to migrating between filesystems. Fortunately, they’ll have a few lingering but ongoing FAT/UFS/extfs rollovers for practice.

OpenZFS Mastery is open for sponsorship at https://sponsor.mwl.io.

December’s Dastardly Sausage

This post goes to Patronizers in December and becomes public in January. Not a Patronizer? You could be! $12 a year gets you my latest updates, occasional free tidbits, and the completely pointless MWL Footnote Fortune File, freshly updated for the new edition of Networking for System Administrators.

A month of paperwork here.

The N4SA2e Kickstarter is “fulfilled,” meaning “Lucas has done everything that he can do” and not “everyone has received their books.” The printer (Ingram) has acknowledged all the drop shipment orders, but they’re backed up for the solstice. I am told that Ingram is running extra shifts to catch up, but physical products take time no matter what you do. This is the second time a campaign went large, and I’ve reached a couple of conclusions.

I am the wrong person to be doing this work.

Chasing a handful of people for their phone number and shipping address? No big deal.

Chasing a few hundred people for same? Big deal.

Some folks handle that just fine, but my brain is wired in such a way that I find it painful. I could just say “eh, they gave me money but didn’t give me an address, not my problem,” but that way leads to unhappy supporters. I want folks coming back for the next book. The photo in this Bluesky post is my goal–folks coming back, book after book, and buying them directly from me. The only way that happens if I make it work transparently. Catching Covid halfway through this fulfillment didn’t help. I need support.

The good news is, my “contact all sponsors and get their correct current shipping address” script worked. I had one misaddressed package. Seems the sponsor runs their own mail server and hadn’t got my email. I can deliver messages to the Email Empire reliably, but one dude on his own? Not so much. Still, I’m calling it a success. That wasn’t so bad.

Dropships are simple, in theory. I prepare a spreadsheet with the hundreds of orders on it. Each spreadsheet has an ISBN, title, address, phone number, shipping method, and so on. I send the spreadsheet to Ingram. They acknowledge it, print the books, and ship. Not a huge job, but tedious and annoying.

I know many detail-oriented people. I could easily hire one for short-term spreadsheet wrangling.

The annoying thing would be training them. Training means explanations. Explanations are best served by documentation. I’ve put off documenting Tilted Windmill Press business processes, in part because the whole time I’m committing them to writing I would have to suppress my fully natural desire to shriek THIS BUSINESS IS RIDICULOUS and while that’s certainly true I’ll eventually run out of air and black out.

But needs must when the devil drives. Capitalism is a pretty senior devil.

The material changes slowly, but too quickly for a book. The sensible way to document my processes in a way accessible to external employees would be… ugh… I don’t want to say it… a (gag) wiki. (I guess I’m now looking for recommendations for simple wiki servers that run well on BSD?)

The logical next question, though? If I’m going to document my processes, and they’re going to be on a web site, should I make the documents public?

Businesswise, I am a unicorn. Very few people make a living writing. Of those, most write for an organization like a publisher or a business. I write books independently and fling them into the public, at scale. A handful of other folks manage it. Huge names? Yes, Brandon Sanderson’s last Kickstarter had 185,341 backers and grossed $41,754,153, so that dude has an entire staff complete with HR department and a full-time spreadsheet wrangler and a single manager to insulate him from all of that. The minuscule gang at my size have all built our own systems for dealing with a hostile industry. Dealing with the vagaries of IngramSpark dropship spreadsheet? There’s maybe a few dozen authors in the world that have to do that for themselves.

But when you first meet the system it feels overwhelming. The newly initiated would welcome documentation.

Ditto for folks who have never used USPS to mail a thousand books from their living room. How do you prepare that spreadsheet? How do you manage the packages? What’s a manifest? It’s a straightforward job, easy to outsource–once you provide instructions.

But if I make this public I would attract comments. “How do I sort columns in LibreOffice?” “Apple Music doesn’t let me export spreadsheets.” “No way a woke doofus like you could have that many readers.” And, of course, the all-purpose “You are wrong.” Yes, I can open each page with a disclaimer and remove all contact information, but people are weirdly persistent in finding a mailbox to complain at.

But when an Ingram spreadsheet first assaulted my eyeballs, I would have cherished an experienced person’s scribbled notes.

But, but, but. Lots of back sides, no front sides.

I’ll be setting up a wiki for internal use, but we’ll have to see if I make it public or not.

The real time and energy savings of hiring someone won’t hit until the second time I hire that same person to help ship books, because the first time I’ll spend more resources documenting and training than I would doing the work myself. Writing this out, I should perhaps hire someone before I have a large campaign. It’s probably better to train them on a Dear Abyss or a Laserblasted than on a viral tech book.

Anyway, what else is going on?

I’m in a new Storybundle that supports World Central Kitchen! That’s cool. My book, Beastly Virtues, will never be in stores. I created this collection just for this Storybundle. I have enough short stories that I can create themed collections on just about anything–well, okay, maybe not for “hope.” Or “joy,” sure, but almost anything! This bundle is all about Wee Beasties, and is a great deal. Digital Reader patronizers, you’ll get a copy of my book but you should definitely check out the bundle.

With N4SA2e out the door, I should have the decks clear to start really cranking on the new OpenZFS book. It’s open for sponsorships, by the way, so do tell your friends. I don’t know if I’ll do a challenge coin for this book, though. Business thrives on predictability, and I don’t know if coins will be available by the time I finish the book or what the tariffs on them will be. I budget about $7-$8 for a coin, which makes print sponsorships slightly less profitable than Kickstarter backers who buy the special edition without the coin. If that becomes $20 a coin, there won’t be a coin. If inflation keeps climbing, there won’t be a coin. I really, REALLY want to not raise the price of sponsorships. I last seriously considered a sponsorship price increase in 2022.

The real answer is that the value of the dollar will continue to degrade until I am forced to raise prices. Then it will degrade further. Because that’s what the dollar does.

N4SA2e was my most sponsored book however, though, and if I can continue to attract numbers of sponsors I can hold off that price increase.

Anyway: docs. N4SA2e. Wee Beasties. OpenZFS. I think that’s it?

Thank you for your support. My Patronizer income might not be huge like some folks, but it’s predictable reliable and steady and that makes a world of difference when it comes to keeping the lights on.

My First Ever Awards Eligibility Post

Every January, novelists announce their previous years’ releases that are eligible for awards. I don’t. The authors offer review copies to award readers. Again, I don’t. Awards give a nice warm fuzzy feeling, but the warm fuzzy feeling I prefer is “having heat and food in frozen Detroit.” I’m good. Peer recognition is great, sure, except for the part where folks notice that you exist.

Many people have designed their careers such that awards might give them a career boost. If you work through a traditional publisher and you win a Hugo or whatever, their marketing team will get you on radio and TV and in big newspapers. People Magazine might notice you exist. That might sell books! Your future books will all get “Award-Winning Author” slapped on the cover.

As I say elsewhere, I lack the infrastructure to leverage awards and have no interest in building that infrastructure on the minuscule chance that I win one. Sign on with a trad publisher? Forget the exploitative contracts and the loss of control; in the time it would take me to sell a book to, say, Tor, I can write, publish, and get paid for four novels (see above re: heat and food).

Sometimes, however, art overtakes reality.

I published one novel in 2025. It demands an awards eligibility declaration.

It started as a joke, but is probably the best novel I’ve written.

Go ahead. Nominate it for a big award. I dare you. I double-dog dare you.

The reviews have all been positive (Goodreads) (Amazon). A uniform five stars.

I triple-dog dare you.

Oh, and because someone will ask: nonfiction book awards? I’ve won one. Others exist, but they don’t help build writing careers. They do help build technical careers, though, so give them to someone they will help and don’t nominate me.