May’s Mandriarchal Sausage

This post went to Patronizers at the beginning of April, and to the public at the beginning of May. Yes, I know, everyone already patronizes me, but if you want me to pay attention sign up here.

As expected, the economy is in an unforced crash. Yippee! Buckle in folks, it’s gonna be a wild ride. I wish to thank all of y’all for backing me. Those folks who signed up to See the Sausage Being Made for $12 a year add up. The whole Patronizer thing often makes me feel like the founder of the world’s most useless cult, but you do keep the bank from taking the house and I am grateful.

I spent some time this month going through my bills. We’ve had the same cable company for almost twenty years and two different houses. (No TV, just a bit pipe.) Every few years, I’d get a letter that they were upgrading my plan. Fine. I gave them a call and told them to cut my bill or lose me. They cut the bill and upgraded my service again. I really need to put “negotiate with cable company” on my calendar for every two years.

Next up: the cell phone. We had Sprint because it worked here, which means we became T-Mobile, which means our bill had gone up. Probably switching to Consumer Cellular. It’s time to stretch those dollars!

Happier news: all active Tilted Windmill Press titles are in Bookvault, which means I will be able to sell them from my bookstore. Yay! I have a box of 21 proofs coming. If they are okay I’ll get them in the store.

I’m already offering the FreeBSD Storage Bundle in print at $24 off. I’m afraid to look at what I’ll have to charge for The Full Michael in print, but it’ll happen.

Okay, fine. Let me build a spreadsheet and add it up.

34 print books, $785. If I give a 20% discount that makes it $628. Call it $629.99. Nobody’s that insane, but if I don’t offer it one of you is gonna ask me why not. Plus, being able to offer it all feels correct. Like the bookstore is truly complete. You want it, I can send it!

Why do a 20% discount? When you buy direct from me, I don’t have to pay a bookstore or a distributor. 20% splits the difference with the reader, which seems fair. I do have paperwork and financial overhead on direct sales, though, so I’m compromising by offering the split on larger orders. Larger orders also save on shipping. Eventually I want to say “buy 4 or more get 20% off” but I implement only one headache at a time.

Sadly no, I can’t include No Starch titles. They are not print-on-demand. I would have to touch them and compute shipping. Not happening.

The question I’m still struggling with is the default format. I have to choose either “print/epub bundle” or “just epub.” Yes, people can change it with the “Format” drop-down. But defaults do impact people. Is it more ethical to set it to the less expensive option? How many people won’t see the “Format” option?

Clearly I need to pivot to “just give me all your money and I’ll decide what you get,” sigh.

I discovered a complication on direct print sales to Europe, though. The printer does not include commercial invoices. Bookvault does not know how much I charged you, or if I charged VAT. You can download an invoice with your order. I had one European customer who had trouble with this, but other Europeans say that’s fairly standard. I need to go through the print store and add that warning to every print version.

The good news is, I have the hard part of the networking book done. Sort of. I sent the TLS chapter to Bob Beck for review, and it went horribly wrong. The good news is, I get to do some new sysadmin stuff and set up QUIC on my web site. Then I get to do some netflow analysis and see how much of the traffic is QUIC versus everything else. Good fun times, for a strange value of good and stranger value of fun.

The good news is, the second half of the rest of N4SA2e requires very little in the way of updates. Some of the warnings need to be louder, but traceroute hasn’t changed, nor netcat.

Laserblasted is due back from copyedit on 15 May, and I’m wondering: if I can crank hard for two weeks, can I finish this draft and get it to tech review by then? I don’t know, but it’s a worthy goal. If nothing else, I’ll have two weeks of cramming on it.

Speaking of Laserblasted: as an experiment, it’ll be exclusive to my store for a few weeks before I release it on other bookstores. It’s a weird book so it’s not a real test of windowing, but I’ve heard from a few folks who missed the Kickstarter and maybe I’ll scoop them up that way.

Laserblasted also taught me a lesson. I had so much fun ranting about the film that I forgot to describe what the book is. Or: I spent so much time saying what it isn’t that I forgot to say what it is.

Meaning that I broke my own rule, and didn’t write the jacket copy before writing the book.

I still am struggling with the jacket copy. I might just steal my pal ZZ Claybourne’s description:

“What would happen if Gomer Pyle got turned into a scathing indictment of the military patriarchal industrial complex but with way more aliens and laser-inflicted explosions than the sitcom managed to achieve? Starring Brad Pitt as Gomer, directed by Sam Raimi.”

I’m scathing? When did this happen?

Er, uh–yeah. I meant to write that. What that says. I planned that. Yep.

Now to get back cover text that means that.

Maybe I’ll just put that on the back.

I really do need to get a Prohibition Orcs Christmas story written for my Christmas collection. Ideally I’ll launch the Twisted Presents Kickstarter on 25 July and be ready to fulfill immediately upon payment, so I can do the N4SA2e Kickstarter right after. Wait–does that work? Let’s sketch this out.

Let’s be pessimistic and say I finish end of May. I want a month for tech review, which means end of June.

I’ll have Laserblasted proofs ready and hard copies ordered before BSDCan, and order the books for delivery when I get home. Fulfill that by the end of June, no problem.

The copyeditor will have N4SA2e in July. She’s usually about six weeks on a tech book. I haven’t checked with her, of course, and she might blow up my entire schedule.

So if I launch Twisted Presents on 21 July and let it run for two weeks, it ends 4 August. I order books 5 August. Yes, I don’t have the money yet, but I’m assuming Kickstarter coughs up the dough on 18 August, like they do. I can ship them that week. It’s only a handful of copies, it’ll only take a day.

So I get the N4SA2e copyedits about 15 August and spend about a week doing corrections and laying out the print and indexing, all that crap. Launch that Kickstarter early September, let it run for three weeks. (Tech book campaigns should run longer than fiction ones.)

Now to work backwards.

Launching Twisted Presents on 21 July means I must have a print proof in hand by that date. I’ll need everything finalized by 7 July. I need the final manuscript end of June. Most of this book was previously published, but the orc tale needs copyediting and everything needs a final proofreading. I might have to use an alternate editor for that, someone less expensive at speed. That’ll take a couple weeks at best, which brings us up to the end of BSDCan. I’m not working on books during BSDCan. The book must go to copyedit by 9 June.

I need a few first readers to read the orc story. That takes two weeks.

I must have a completed orc story by 26 May. Achievable.

Except I’m cramming to finish N4SA2e by then.

Have I ever mentioned that I’m terrible at scheduling?

All of this is built on the very slender reed of finishing N4SA2e by the end of May. Which means I get QUIC on my web server.

I think I better go get to work.

91: Vice Without a Biological Limit

Networking for Systems Administrators is out for tech review, so I’m working on my forthcoming Christmas collection. Here’s a snippet from Twisted Presents.

Money is the one vice without a biological limit. We can gorge on food and wine until we puke, and the feast ends. Do enough drugs and you die. Despite what teenagers think, you can’t have sex twenty-four seven. Vices are self-correcting.

But if you’re greedy enough, you can pile up money forever.

And you can teach your children that your vice is their birthright.

The only reason I don’t say “eat the rich” is because toxic metals accumulate at the top of the food chain.

But even among the rich, there’s some I hate more than others. Until today, “people who scam charities” weren’t even on the list. I mean, the biggest charities in the world spend eighty percent of their donations on administration. They’re just another face for the oligarchy.
But a meager fraction of charities do charitable work.

There are wholesome people who honestly want to help others, and invest their time and effort and meager income towards that.

Some of those handle large amounts of money, and spend that money on their cause.

But every dollar spent helping the helpless isn’t invested in cybersecurity.

If you have the skills, and totally lack decency, these charities are easy targets.

Twisted Presents will be coming to Kickstarter in July, so I can get it to you well before Christmas. For twelve days, of course.

Buy Your Paperbacks Directly From Me

All Tilted Windmill Press titles are now available directly from me in paperback and ebook at https://tiltedwindmillpress.com. All paperback purchases include the ebook. You’ll get the ebook immediately1, and the print will arrive in a week or so.

Books will be printed in the US, Canada, UK, and Australia. This reduces both shipping costs and environmental impact. Books aren’t exactly green, but local printing makes them less brown. (Are ebooks greener? That’s a great argument over a drink.)

I am excited beyond words. I have been working towards this ever since my first book came out in 1992.

Benefits to you? Those bundles I offer, like the FreeBSD Storage Mastery bundle? There’s now a discount print version. That ridiculous The Full Michael bundle that includes everything I’ve indie published? You can now buy the whole thing in paperback.

Do I expect anyone to drop $624 on a stack of books? No. But I am delighted to have that degree of control.

Books from No Starch Press (Absolute FreeBSD, Absolute OpenBSD, and Network Flow Analysis) are not included. Sorry. I don’t have the access to ship those touch-free on demand. The ILUVMICHAEL coupon code still gets you 30% off at their site and gives me a couple bucks extra, though!

Completing this was a huge amount of work, but the publishing industry is doing its best to eat writers alive. The only way to survive is disintermediation.

I haven’t made hardcovers available yet. Hardcover sales are minuscule next to paperbacks. Some books present challenges, and I’m not sure selling them direct is worth it. I’m doing the easy hardcovers first in the hope that inspiration strikes.

Future books will be released in on my site a month before they’re available at retailers. If they’re trying to eat my career, I see no reason to prioritize them.

90: Demands a Little Madness

Now that Laserblasted is available in my store, here’s one final snippet from that book.

For all you might hear about stories of weapons haunted by previous owners, there aren’t that many actual legends. The modern archetype would be the Elric saga, overflowing with “I’m suddenly strong and my body’s acting weird and I keep breaking things please help me oops I’m doomed so I’m taking all of you with me.” There’s no better metaphor for a young boy transitioning to adulthood. Stormbringer isn’t exactly haunted, only ravenous for everything it can devour—again, like teenage boys. The Norse legend of Tyrfing teaches the essential lesson of “if you’re going to kidnap me and make me forge a mighty magic blade for you, you better expect that magic to fuck you up something fierce.” Master Japanese swordsmith Muramasa was rumored to pass his blood thirst and madness into his blades. Even unenchanted swords made by mentally healthy swordsmiths are viciously dangerous. Merely training with live steel, especially before antibiotics, demands a little madness.

If you buy the print from me you get the ebook free. Or you can wait a month to get either version from other, lesser bookstores.

“Networking for System Administrators” sponsorships closing and schedule.

Yesterday I finished a raw draft of the new Networking for System Administrators. It’s not ready for technical review yet; the engine has all the pieces, but there are loose bolts everywhere and a couple of the belts are repurposed nylons. I’ll get it out for tech review this weekend.

On 1 June 2025, I close sponsorships. If you want to sponsor it, this week is your last chance. I promised to do a challenge coin for print sponsors and Patronizers so I will, but the next one probably won’t. I’ll happily absorb $10 per sponsor to do something daft, but not the $25 the US’ Wheel of Tariffs threatens. (Regardless of your politics, unpredictability is death to business.)

The tentative schedule for N4SA2e is:

  • June: Technical Review
  • July-August: Copyedit
  • September: Kickstarter
  • October: ship sponsor and Patronizer copies, both print and ebook exclusive to tiltedwindmillpress.com
  • November: standard retail release

The print version will come in a special backer-exclusive edition available only to print sponsors, Patronizers, and Kickstarter backers. (Kickstarter backers can’t get the challenge coin; that’s exclusive to early backers.) I can’t say it’ll be as daft as Ruin Your Mail By Running It Yourself or the Networknomicon, but it will exist.

Then again, I always think my special editions are lame. You can make your own opinion.

89: Cheap Cat5 Cable

I’m grinding on the new Networking for System Administrators so here’s a chunk.

Ultimately, the Internet is a bunch of routers, switches, firewalls (however you define them), and other devices that connect a tangle of cables. Once a client request traverses the local WiFi connection, it travels through a bunch of wires and devices until they reach your server. Ultimately, every Internet node is connected by wire that can be traced from the local café to downtown, where it joins a bigger cable that goes across the country, perhaps joining a huge cable that runs under an ocean or three to reach another continent. That huge cable gets broken up into finer and finer wires until it finally reaches the cheap CAT5 that connects the server to the patch panel. Some parts of this link might run over satellite connections or carrier pigeons or who-knows-what. Every one of these components is fragile.

It’s a miracle the Internet works. At all.

When I finish this draft and get the book to tech review, sponsorships will close. If you want your name in the book or the challenge coin, grab it now. And when the Laserblasted print arrives in my store, I’ll be reading one last tibdit from it. If you prefer ebook you can get it now.

“Networking for System Administrators” restructuring

No, not the book this time. The product. Previously you picked a format, print or ebook. If you sponsored for print, Woocommerce used your address to calculate shipping. Cool. It took me a couple iterations to get that working, but it’s the way the rest of the world works.

Then I added print books via BookVault.

Turns out that Woocommerce does not like multiple shipping systems. It says it’s fine. It is not. After months of fighting with this, I realized that my attachment to sponsor shipping autocalculation was causing pain. I have restructured the product so that you choose a destination and pay accordingly.

The total price has not changed. The list price is now shipping-inclusive to avoid Woo’s clunky shipping system, that’s all. While sponsorship is an especially terrible deal for my Australian backers, it is no more terrible than before.

I’m still pushing to get the first draft of this book finished by the end of the month.

Also: attachment is the source of all pain. Well, that and blunt instruments. Those hurt, too.

Laserblasted Update

My copyeditor got the manuscript back to me last weekend. I’ll be getting it into production this week and next, amidst finishing the new Networking for Systems Administrators. Once the book can be purchased both print and ebook will be exclusive to my store for a month or so, then I’ll release it to the wider public.

I was hanging out with ZZ Claybourne and a couple friends, so we picked the movie we’re going to watch and review for the Kickstarter stretch goal. It is… drum roll, please…

Evil Brain from Outer Space.

I see no way this will end well.

88: The Same Bucket

The garbage truck that exploded outside my house means I am karmically bound to share a piece of the networking book.

Authoritative nameservers contain the information for one or more specific domains. I run authoritative DNS servers for my domains, such as mwl.io and prohibitionorcs.com. Anyone in the world who performs DNS queries on my domains gets their authoritative answers only from my servers.

Recursive nameservers provide DNS lookups for clients. When you browse to my web site your computer asks a recursive nameserver for the IP address to connect to. The recursive nameserver finds the authoritative nameserver for the destination site, queries it, and returns the answer to your computer.

Put your authoritative and recursive nameservers on different hosts. The twentieth-century practice of combining authoritative and recursive DNS on one server led to many security problems. In hindsight, the “store the sacrosanct Single Source of Truth for our company’s public face” function and the “collect and cache random data from any system anywhere on the Internet” function should not share the same bucket.

Remember folks, don’t throw lithium-ion batteries in the trash! But do sponsor Networking for System Administrators.

Migrating from Apache 2.4 to Caddy

I’ve been using Apache since the 1990s. The networking book requires information about QUIC, so I need experience with QUIC, so I need HTTP/3, so I can’t use Apache.

I experimented with Caddy on my test host. It worked well as a reverse proxy, so I began putting it in place in production this weekend. (If you deploy Caddy, definitely have it run as a user other than root!)

As I went through the docs to prepare, though, I realized that not only would it would be less complex and more robust to drop Apache and use Caddy, it would also be easier.

My Apache configuration files are large and complex because Apache can do anything. I don’t need a web server that can do anything. I need a web server that serves static files, talks to php-fpm, and supports TLS. The Caddy docs are complete, but I didn’t find a simple guide for what I wanted to do, so I’m posting this. I suspect that guide exists but is buried beneath pages of search engine poison.

This uses Caddy 2.9.1 on FreeBSD 14. My config files are in /usr/local/etc/caddy, symlinked to /etc/caddy.

The main config file, /etc/caddy/Caddyfile contains only:

import sites/*conf

The /etc/caddy/sites directory contains each of my sites in its own file. Mostly.

Here’s one of my old sites in blackhelicopters.org.conf:

blackhelicopters.org www.blackhelicopters.org {
        root * /var/www/bh
        file_server

        log {
                output file /var/log/bh/bh-caddy.log
                format json
        }
}

The first entry on the first line is the server’s main name, blackhelicopters.org. (I could probably let that domain go, but my oldest friends have that email in their address books and it’s worth a couple bucks a year to not inconvenience them.) The following hostnames are what Apache would call ServerAlias entries: other names this host responds to. Every name here goes into the X.509 certificate.

The root statement tells Caddy where to find the files for this site. Every URL goes under here. If I had Apache Directory statements, I could put them here.

The file_server statement means “hand out files.”

Last, there’s a logging statement. Caddy logs are written in JSON, making them harder to eyeball but easier to mechanically parse. Pipe the logs through jq(1) to read the parts you want.

Several of my domains exist only as legacy redirects. While https://michaelwlucas.com and https://michaelwarrenlucas.com made sense in a keyboard-centric era, they’re a pain to type on a phone.

blather.michaelwlucas.com www.michaelwarrenlucas.com michaelwarrenlucas.com www.michaelwlucas.com michaelwlucas.com mwlucas.org www.mwlucas.org {
        redir https://mwl.io
        }

This config doesn’t even serve files. It’s like setting DocumentRoot to /var/empty. Any traffic to these hostnames should be redirected to my current web site.

So what about that all-important main web site?

mwl.io  www.mwl.io {
        tls mwl@mwl.io

        log {
                output file /var/log/mwl/io-caddy.log
                format json
        }

        root * /var/www/io
        file_server
        php_fastcgi localhost:9000

        @disallowed {
                path /xmlrpc.php
                path *.sql
                path /wp-content/uploads/*.php
                path *~
        }

        rewrite @disallowed 'index.php'

        redir "/ks" https://www.kickstarter.com/projects/mwlucas/mwls-next-1-april-book"
...
}

the tls statement puts my email address in the Let’s Encrypt certificate request. I should probably go back and add that to the sites I did earlier.

The php_fastcgi option tells Caddy where to find the php-fpm engine.

The @disallowed statement defines a list named “disallowed.” The following rewrite statement transforms requests to files with those names, redirecting them to the index.

Finally, I have several redirect statements for my convenience.

Test a configuration by going to /etc/caddy and running caddy validate, much like apachectl configtest. The configuration files are JSON, so the parser isn’t quite as straightforward as you might expect.

# caddy validate
2025/05/05 15:02:38.489 INFO using adjacent Caddyfile
2025/05/05 15:02:38.489 INFO using config from file {"file": "Caddyfile"}
Error: adapting config using caddyfile: /usr/local/etc/caddy/sites/test-twp.conf:1: unrecognized directive: test.tiltedwindmillpress.com
Did you mean to define a second site? If so, you must use curly braces around each site to separate their configurations.

Here’s the problem: the error is not where it says the error is. The error is before the cited point. The sensible thing to do is to test after creating each site’s configuration file. If you get bored and do all your sites while watching reruns of Adam and Jamie welding JATO units to a hamster ball so they can replicate that urban legend about the Syria-Guam War, you’ll have to do a binary search of your files to see where the problem is. Test each one as you finish it.

Once you have a parseable configuration, shut off Apache and start Caddy. Watch /var/log/caddy/caddy.log for errors. Test all of your sites.

Am I happy with Caddy? Yes, so far. Am I keeping my known-working Apache configuration around? Also yes, so far. If I suffer an attack of the AI scrapers, I might need to fall back to a Caddy reverse proxy so that I can implement Anubis. Yes, there’s an Anubis Caddy module but it’s a proof-of-concept.

What kind of impact has Caddy had on my site? It seems faster, but that might be QUIC aka HTTP/3 rather than any difference between Caddy and Apache. Of course, QUIC is a difference between the two. How much of my traffic is QUIC now? QUIC runs on UDP port 443. First, let’s check how much traffic went to and from port 443 yesterday, on all protocols.

# nfdump -R . -B ip 23.139.82.3 and port 443
...
Summary: total flows: 58605, total bytes: 6.9 G, total packets: 7.0 M, avg bps: 1.0 M, avg pps: 127, avg bpp: 990
Time window: 2025-05-04 00:00:00 - 2025-05-04 23:59:59

6.9 GB. How much of that is UDP?

# nfdump -R . -B ip 23.139.82.3 and port 443 and proto udp | tail -4
Summary: total flows: 1620, total bytes: 428.4 M, total packets: 412444, avg bps: 62756, avg pps: 7, avg bpp: 1038
Time window: 2025-05-04 00:00:00 - 2025-05-04 23:59:59
Total flows processed: 750342, passed: 1620, Blocks skipped: 0, Bytes read: 66537172
Sys: 0.0209s User: 0.0209s Wall: 0.0399s flows/second: 18806544.9 Runtime: 0.0423s

428.4 Mb of my traffic is QUIC? Firefox and Chrome derivatives both use QUIC if available. The only clients that should be using TCP are stupid bots and crawlers–

Oh. Maybe I do need to implement Anubis. Dammit.