Today, I shipped the physical rewards for the Devotion and Corrosion Kickstarter. The Post Office was kind enough to use a tote to deliver a beat-up package. Today I’m returning it.
These books arrived at my house a couple days after I opened sponsorships on the mail book. Unfortunately, in those intervening days I caught covid. (Zero stars, recommend catching distemper or cercospora leaf rot instead.) I’m recovering, but this is my first effort at working since I fell ill.
Even with that interruption, fulfilling this Kickstarter went much better than fulfilling the Prohibition Orcs one. In the future, I will have fulfillment ready to fire as soon as Kickstarter’s payment hits my account. SO much less stress that way!
I have also made an early draft of Chapter 0 available to help explain what the book is and is not. Before you start sending me corrections, check the disclaimer at the top.
Despite inflation, I have not raised the price of sponsorships. I have passed through the shipping charge for print sponsors, however, so I’ve totally raised the price. But it’s transparent, and relative to where you live. Note that the shipping is per-continent, not calculated live. I can’t calculate shipping for something that doesn’t even exist yet.
And yes, I’m building a Kickstarter into this book’s production plan. That will be more of a pre-order, although I do have some thoughts on how I can use Kickstarter to enhance the book. The Kickstarter will replace a direct pre-order, as an experiment.
Running your own email is a struggle against an opponent you can’t see. That’s why the cover is a take on Battle of Vimy Ridge. You’ll be able to buy Eddie Sharam’s original painting, either through the Kickstarter or direct auction. It will be a full wraparound cover, but here’s the first draft of the front.
Now, if you’ll pardon me, I have to go shell gmail. Uh, figure out why they’re not taking my mails, I mean.
This is uncopyedited. Unreviewed. It’s what we call a “vomit draft.” It exists to illustrate scope, not for folks to send me corrections. Yet. Before I complete the book, I will probably discard and redraft this chapter. Bridge Out Ahead. Slippery When Wet. No warranty of fitness for purpose. Probably causes cancer, chin hemorrhoids, and alternaria leaf blight. (Updated 2023-04-06)
Chapter 0: Introduction
Email is the beating heart of the Internet. Yes, over the last couple of decades online forums and social media and chat systems have blossomed and thrived, but we keep returning to email. Email works everywhere with an Internet connection, and many places without. Unlike chats, where the other people involved see that you’ve received their message and are choosing to ignore them, you answer email when you feel like it. Your correspondents don’t get a notification that you’re typing a reply and watch the screen until you compose words. Email is one of the few surviving asynchronous communication tools, and is available on every platform without vendor lock-in.
The first email-like message was sent in 1965 at MIT, when the Internet was only a drunken dream of comp sci grad students. When those ragtag students finally achieved a network, they developed the primordial Internet protocols. Today’s email looks nothing like theirs—for one, email addresses have those esoteric @ symbols in them rather than nice simple exclamation points. Modern email is built on those primordial protocols and the lessons learned from them, however. That’s not bad—all broadly used protocols evolved the same way. Today, a handful of providers dominate email services. Companies like Google and Microsoft control most email addresses. They’re an Email Empire, while those of us who run out own email form our own ragtag band struggling for independence.
It can be done. The protocols are comprehensible, the software freely available, the debugging tools comprehensible, and the hardware affordable.
The problem with running email has nothing to do with the technology. With the right knowledge, you can set up an email server in moments. The problem is that the email server is only one part of running an email server.
Consider setting up your first web site from scratch. You install the software, start the server, and point your web browser at the host’s IP address. You get the default page and a warm thrill of victory. You have a web server! If you put content on it, you can tell your colleagues “Hey, browse to 203.0.113.99 and you’ll find my new web site!” It’s a thing. Add an entry in the Domain Name Service (DNS) and you can offer folks a link like http://mwl.io. More testing and you add TLS, granting you the precious S in https. Step by step, you build a web site that complies with modern standards.[1]
Email isn’t like that.
Running email in the real world is not configuration problem. It is about citizenship. Once the Email Empire decides that your server is inadequate or untrustworthy, it’s very hard to get them to change their mind. You need to solidly participate in the email society before sending your first email to the outside world. Sure, you can set up a couple of throwaway hosts on expendable domains and and bounce email exclusively between them to see how the software works, but if you want to run a real email server and you want other servers to accept your email, you must quickly establish a good reputation.
Once you establish your system as a good citizen that follows standard practices, however, operating a mail server is comparatively low maintenance. Most of the time I spend on my email server is spent applying security patches, same as any other Internet-facing application. Running a mail server for a large corporation is a full time job, yes, but most of that is spent explaining to users that a bounce message of “The recipient’s email account is over quota” means that the email bounced because the recipient’s email account is over quota.
Run Your Own Mail Server uses common freely available software to illustrate the mechanics of serving email, but the book’s main focus is on establishing that citizenship. You can use any software you like that fulfills the necessary roles.
Why Bother?
If email is such a pain, why run it? Privacy, control, and education.
Service providers often scan email for advertising keywords or saleable personal data. While ad blockers keep the ads from your face, they solve the problem at the wrong end. Yes, if you email someone in the Email Empire address the Empire can tie the content to their records about your email address, but when you correspond with other outsiders they can’t. Email can be intercepted, but that interception takes work.
Privacy is especially important in business. A service provider might promise to not mine your company emails, but one of their sysadmins trying to fix your issues might have to examine your email. A long time ago, at an employer far far away, I had to troubleshoot a server problem caused by a user innocently attaching a word processor document to a mail. That was its own special fun, but—I had to examine the characters in the document to solve the problem. I saw parts of the document. While most of email’s rough edges have been burnished off since, violating privacy during troubleshooting remains a real possibility.
You control your own email. Many service providers offer fancy dashboards that let you add and delete users or set quotas, but when something goes sideways you’re stuck working with their support department. The quickest way to identify a problem is to check the log. Waiting for a support tech to answer your trouble ticket is infuriating. When you run your own email, you control the problem.
Perhaps the least valid but most important reason for many is pure “geek cred.” We want to know how things work. Just as building your own firewall is a great way to learn about networking, running your own email server is the best way to understand this ecosystem. You run your own web server, DNS server, cloud storage and home automation, why not run your own email? You can, and you will learn a lot.
Prerequisites
Operating an email server is not a task for a beginner sysadmin. You must have certain skills before you start.
You must have used email. You must be aware of things like email addresses, attachments, and related concepts.
You must know how to manage your operating system. You need to back up your servers, apply security updates, and in general make your hosts fit to sit on the naked Internet. This book assumes and recommends open source software. I demonstrate everything with Postfix on FreeBSD, but provide the context for all my configuration choices so you can make your own decisions on any platform.
You need at least two test hosts attached to the Internet. These could be minimal virtual machines out in the cloud, hosts in your lab, or whatever. Tests inflict negligible hardware load. These hosts need different but static IP addresses. You must also control the world’s access to those addresses, as email requires opening several TCP/IP ports to and from the servers. Most home Internet providers specifically block these ports.
Those hosts should be in different domains. Yes, you could send email between newyork.solveamurder.org and detroit.solveamurder.org, but your brain absorbs better patterns if you test between the wholly unrelated solveamurder.org and ratoperatedvehicle.com. Domains are inexpensive these days. Get two—or set up private DNS and test between example.org and example.net.
You must control the forward and reverse DNS for your test hosts. Supporting email for a domain requires creating and maintaining several DNS entries in that domain. Similarly, the reverse DNS must match the hostname and the forward DNS. If your host advertises its name as mail.solveamurder.org, but reverse DNS identifies your host as customer87.chicago.bighosting.com, your email will go nowhere. You must have correct reverse DNS on all IPv4 and IPv6 addresses. Follow best practices and have separate authoritative and recursive nameservers. The recursive nameservers should validate DNSSEC.
An understanding of basic public key cryptography is also essential. You don’t need to grind through Diffie-Hellman calculations by hand, but you do need to know the difference between public and private keys and how they’re used. You must routinely protect private keys. Similarly, you need a basic understanding of TLS. Users must authenticate to check their email, and sending usernames and passwords in clear text across the Internet is unwise. The X.509 certificates needed for TLS used to be very expensive, so by tradition SMTP does not require TLS. Certificates prices have plunged to zero, so there’s no longer an excuse to not use TLS. This book uses Let’s Encrypt certificates.
We will also use outside services to support email troubleshooting. These are all services that require writing custom code, a skill orthogonal to running a mail server. I’m certain that immediately after this book comes out, someone will release a package that lets us easily handle these features on our own systems.
If you lack any of these skills, permit me to recommend my books Absolute FreeBSD, 3rd Edition (No Starch Press, 2018), Networking for System Administrators (Tilted Windmill Press, 2015), and TLS Mastery (Tilted Windmill Press, 2021). I also recommend Cricket Liu’s DNS and BIND (O’Reilly, 2006)[2] and, once you’ve finished that, my own DNSSEC Mastery, 2nd Edition (2022).
From now on, I will toss around terms like network port and X.509 certificate and floccinaucinihilipilification and expect you to either know what they mean or how to look them up. You cannot suffer from hippopotomonstrosesquippedaliophobia and run an email server.
Email Tools In This Book
Email isn’t a server, it’s a system with several components. The most obvious is a “mail server.” The mail server is a host that accepts email from users and exchanges mail with other servers. Most of the system speaks the Simple Mail Transport Protocol (SMTP). This book uses Postfix (https://www.postfix.org) as a reference platform.
Your users would probably like an option to check their email from their desktop. This might use a mail client like Thunderbird, using a protocol like IMAP. We’ll demonstrate clients and IMAP with Dovecot. Other users prefer a web interface, so we’ll discuss <<<pick one that speaks imap>>>. We’ll debug with mutt (https://www.mutt.org).
Once you have the core system working and can send email locally, you’ll need basic antispam protections. We’ll discuss greylisting and DNSBLs.
You can now risk telling the world about your mail system via DNS. This routes us into topics like Sender Policy Framework (SPF), and Domain Keys for Internet Mail (DKIM). With those basics you can look at acronyms like DMARC, BIMI, MTA-STS, and TLS-RPT. The main goal of this book is to teach you about these tools and protocols so that you can improve deliverability of your email.
We will focus on modern email. While some networks still rely on UUCP and bangpaths and POP3, those are vanishingly rare. Don’t deploy them on the modern Internet.[3] We will also ignore proprietary replacements for these protocols. Many mail service providers offer a convenient API that replaces SMTP or IMAP. There’s nothing wrong with using these if you’re willing to accept the vendor lock-in, but this book doesn’t cover them.
Spam
Unsolicited advertising is the scourge of email. While you’ll hear it called things like unsolicited commercial email (UCE) or junk, the most common technical term is “spam” because the Internet was built to a Monty Python soundtrack.
For about thirty years, people considered email a miracle. You could write a message at your desk and hours or even minutes later it would reach the other side of the planet. The early Internet was populated by computer science university students, select computer-related companies, and the US military. Blissfully free of broader commercial interests, SMTP was designed for that open network. Sysadmins ran open mail relays for the convenience of other sysadmins who suffered outages. Everyone worked together to sustain the miracle, with little concern about abuse.
In 1978 Gary Thuerk, an employee of the Digital Equipment Corporation, emailed several hundred people he didn’t know an invitation to a demonstration of the new DEC packet-switching systems. Nobody had ever tried commercial advertising over email. Reactions were overwhelmingly negative, but reasoned. The broader community agreed that commercial advertisements were unacceptable. Five days after the message went out the chief of the US Air Force’s ARPAnet Management Branch, Major Raymond Czahor, called Thuerk’s boss to tell them to never do it again on pain of disconnection.
This first advertising email sold twelve million dollars of computers.
Thuerk’s email established the “do not advertise in email” precedent. It also sold twelve million dollars of servers, establishing the “random email marketing is highly profitable” precedent. Going forward, the network relied on the social contract and Air Force majors to prevent junk mail. Email protocols remained unchanged.
In 1990, UUnet started building its own commercial IP network. UUnet’s AlterNet could communicate with the Internet, but was not bound by government rules. While people that run businesses might be fine, most corporations are indifferent to feeble social contracts and do not answer to the military. The first spam hit Usenet in April 1994, and spam email followed immediately afterwards.
The first spam hit like an unexpected toxic spill at the kindergarten playground. Mail administrators panicked, implemented short-term fixes, and quickly learned that a protocol optimized for deliverability was unable to stop delivering messages. Sysadmins blocked junk mail domains, so mailers forged their originating domain. Blocking spammers’ networks led the spammers to relay their junk through well-meaning third parties or hacking into unrelated but unpatched mail servers. Despite spending time and money abusing the protocol, spammers made money.
Unchecked spam makes email useless. Almost every change in SMTP in the last thirty years has been propelled by the need to eliminate spam while retaining broad interoperability.
Opt-in email marketing is fine. If you want your favorite creator to tell you when their next big thing escapes, great! You should get those emails until you opt out. I live and work in Detroit, though, and the dude that keeps trying to sell me courses on South African taxation law can take a long walk out of a short airlock.
If you wonder why email-related software works as it does, or why we have all these add-on protocols, the answer is almost certainly “spam.”
Deliverability
Your users expect their email to reach the intended recipient. Deliverability is how well mail from your system fulfills this expectation. Before spam, deliverability wasn’t a huge issue. In the 1970s and early 1980s you might have to route your email through U of C Berkeley to MIT to the Imperial College of London to TAMK via overloaded phone lines, but it was a well-understood problem. The decade of 1983 to 1993, after the standardization of the Simple Mail Transfer Protocol but before spam, was a golden age of email connectivity.
If mail from your server looks like spam, the destination server will discard it. Your email is not deliverable. As all mail systems have different spam-identification systems configured in different ways, deliverability varies by domain. The goal of this book is to help you make your email as deliverable as possible by configuring protocols that allow recipients to verify your mail’s authenticity.
Even with all of these defensive and authenticy protocols configured properly, you can still trash your deliverability by sending spam.
IP Addresses and Email
“I need full control of a test machine on the public Internet? I’ll go to my favorite provider and spin up a VM!”
Not so fast.
In the email ecosystem, IP addresses have reputations. Addresses previously used by spammers are distrusted. Anyone with an email address and a credit card can create a virtual machine in moments. Spammers have no problem setting up a cheap VM and spewing garbage into everyone’s inbox. While this violates every hosting company’s Terms of Service, spammers don’t care.
One of the tools mail administrators use to block spam is blocking network addresses of known spammers. Hosting provider IP addresses have almost certainly been used before. If a spammer previously used your address, it’s probably on one or more block lists of known bad addresses (Chapter XXX). Almost everyone uses at least one block list. Your email is undeliverable before you start.
With proper automation, a spammer can send millions of emails before the provider’s Network Operations Center can shut them down. A hosting provider will receive dozens or hundreds of complaints within minutes of this flood. While most providers shut down spammers quickly, those complaints cause extra work. Many providers will send lists of the IP addresses that they provide to customers to block list maintainers, so that those systems will proactively block email from those addresses.
Other providers take no action against spammers. Their terms of service declare that spam is not allowed, but they do not enforce it. Block list maintainers quickly identify IP addresses assigned to these providers and add them to their lists. No matter how much you intend to be a good citizen, any email server on these providers will have poor deliverability.
Before choosing a hosting provider for your email systems, ask around to see if other people run email servers on that provider and if that email is broadly deliverable. If you have no contacts who run mail servers, go ahead and try on your favorite provider but be prepared to move. I will not offer recommendations, as they will be obsolete within months.
Perhaps your organization has IP addresses issued by an ISP, or owns its own addresses. A previous owner could have soiled those addresses. Also, some Internet Service Providers block outgoing email at their network border. Check these before starting.
If your addresses have never been used for email, they have no reputation. “No reputation” does not mean that they are trusted, merely that they are not untrusted. You get to make your own reputation. This book will help you do it well.
If you have no control over your test system’s location, you can still educate yourself but remember that your tests will be limited in scope.
Email Components
If you’re accustomed to web-based email, you’ve probably picked up how mail works. When you send an email your web browser talks to the mail server. That mail server talks to the recipient’s mail server. When the human wants to read the email, they open their browser and see it.
If you use a desktop email client you know it’s a little more complex. You compose a message on the client and send it to your mail server. Your mail server sends the message to the recipient’s server. When the recipient opens their email client, the client downloads the email.
Even that’s a shallow view, however. Email was designed by actual computer scientists, and email systems have an actual architecture as documented in RFC 5598. Every email you send flows through the system. Here’s a slightly less shallow view.
ryoms fig 1-1 message flow through email
Any email user interface is called the Mail User Agent (MUA). The MUA is most often called the mail client. It might be Thunderbird, a web page, or part of an office suite. You can even use a program like netcat to manually send mail to the server. (There are also command-line programs that you can run on the mail server itself, like mutt. They unfairly leverage their location, so we will ignore them for now.) When you send the email, the MUA transmits it to the Mail Submission Agent via SMTP. Usually, but not always, the client copies the message to the Message Store.
The Message Store (MS) is the server’s copy of the user’s emails. Most Message Stores retain a copy of both the ingoing and outgoing messages, at least until the user deletes them. Mail clients use the Internet Message Access Protocol (IMAP) or the older Post Office Protocol (POP3) to synchronize their messages with the server. On Unix systems the Message Store is often in the user’s home directory, something like /home/mwl/Mail.
The Mail Submission Agent (MSA) runs on a server. It accepts mail from mail clients and send it to the Mail Transfer Agent. The MSA communicates with SMTP.
The Mail Transfer Agent (MTA) receives emails from the Message Submission Agent and other Mail Transfer Agents, identifies their destination, and forwards them. The sender’s MTA stores outgoing email in a mail spool or a mail queue and attempts to identify and contact the recipient’s MTA. The two servers have a dialog over SMTP that boils down to “Hi, I have an email for one of your users.” “Yes, that’s my user, please send the mail.” You might hear the MTA called a mail exchange or an MX.
A MTA that receives a message forwards it to the Message Delivery Agent (MDA), sometimes called the Local Delivery Agent (LDA). The MDA accepts the message and stores it safely on disk. If you’re accustomed to managing Unix systems, the MDA places user messages in /var/mail. The MDA also handles tasks like server-side email forwarding, automated out-of-office messages, and filtering email into folders.
When the recipient’s MUA checks for new email and downloads it, it usually moves messages into the recipient’s Message Store. The recipient’s MUA might further sort the messages, delete certain messages, or any other sort of processing configured by the user.
Finally, the user’s mail client downloads the mail. The email is complete.
Out of all these components, which is the “mail server?” None of them, and all. Each component might be on a separate machine, or any of them could be combined with their peers. A single organization might have different MTAs for receiving and sending email. It depends entirely on the system load, the number of users, and which headaches each organization prefers. A user might call the Message Store or the MSA the mail server. Outside organizations might think that your MTAs that receive email from the outside world are your mail servers, unless they’re having trouble receiving emails from you and realize that you have entirely different MTAs for outbound mail. Even if you have only one server that controls all of these roles, stop using the phrase “mail server.” Look more deeply.
This book avoids saying mail server, except when I want to confuse you.
Email Security
Before spam, email clients authenticated only when connecting to the message store. Clients did not need to authenticate to send mail to the mail submission agent. Mail submission agents could dump email straight into the MTA. MTAs did not verify each others’ identity, and could often fling messages at any server that looked suitable. An unprivileged user could trivially forge emails from tax agencies. Everybody knew that email was public, and that someone with a carefully placed packet sniffer could capture messages.
The appearance of spam made everyone say “Maybe we should lock down these other connections?”
The server-to-server protocol is still called Simple Mail Transfer Protocol, but it’s much less simple. It now supports TLS. It uses different TCP/IP ports depending on which piece of the system you’re talking to. The protocol commands are still plain text, however, and can be replicated by a sysadmin with netcat and knowledge. Even wrapping the connections in TLS only prevents eavesdropping in transit.
As with everything in computing, these protections are not perfect. TLS can be broken, through rogue Certificate Authorities and misconfiguration and good old-fashioned theft of private keys. Clever people can figure out how to violate the underlying protocols. You need to keep up with routine system maintenance.
SMTP itself is not enough to successfully exchange mail with another network, however. Email has added protocols that help secure, restrict, and authenticate email. Most of this book discusses those ancillary protocols. We’ll cover them briefly here to orient you, then delve deep into each.
DNS
The Domain Name Service maps hostnames and IP addresses to one another. You must have solid control of your DNS to successfully run email. Domains publish their MTAs in DNS, and many of email’s ancillary security protocols provide their information in DNS.
This raises the question of DNS security. Like email, DNS was created when the Internet was largely a private network. Also like email, security was integrated afterwards. DNS Security Extensions (DNSSEC) are today’s standard for maintaining the integrity and authenticity of DNS information, but have not yet been widely deployed.
Every domain that receives email should publish a Mail Exchanger (MX) record in DNS, telling remote MTAs the hostnames of the domain’s own receiving MTAs. You’ll need additional DNS records for each protocol you deploy.
Sender Policy Framework (SPF)
Where the MX record tells the world which hosts receive a domain’s email, the Sender Policy Framework (SPF) tells the world which hosts are authorized to transmit mail for a domain. SPF declares the hostnames and networks of outbound MTAs, not clients or mail submission agents. Modern SPF is distributed via DNS TXT records.
SPF syntax lets you declare who can send mail. You can make rules like “only these addresses,” “any of the MTAs for this other domain,” or even “this domain does not send mail.” For example, the SPF record for my domain mwl.io contains a few IP addresses and a hostname.
When a MTA receives a connection claiming to have mail from a particular domain, the MTA checks the SPF against the incoming machine’s IP address. If they don’t match, the connection is dubious and probably should be rejected. If someone sends email from mwl.io to a Gmail user, Gmail’s server checks the SPF record for mwl.io. If the forger’s IP address is not in my domain’s SPF record, Gmail drops the connection and records that the forger’s IP is untrustworthy.[4]
SPF is considered the lowest level of authenticity, and is easy to implement. It requires no maintenance, until you add a new outbound MTA.
DomainKeys Identified Mail (DKIM)
Where SPF declares where email comes from, DomainKeys Identified Mail (DKIM) authenticates the email’s sender. DKIM information is published in DNS TXT records tied to a _domainkey record.
DKIM lets an organization publish a policy on how to authenticate mail from the domain. You can tell receivers to validate or ignore SPF, to report or ignore DKIM failures, and how to test messages.
DKIM is more difficult to implement than SPF, but not difficult. Best practices say you should rotate your keys regularly, so it does require more maintenance.
Domain-based Message Authentication, Reporting, and Conformance (DMARC)
SPF and then DKIM improved mail authentication. Domain-based Message Authentication, Reporting, and Conformance (DMARC) is the next evolution. It uses SPF to identify legitimate mail sources. It uses DKIM to authenticate email. Its robust policy language lets organizations define how recipients can identify legitimate mail and report failures in those two protocols.
DMARC records are distributed via TXT records, tied to a _dmarc record.
Brand Indicators for Message Identification (BIMI)
If your organization uses a trademark for marketing, you might have an interest in Brand Indicators for Message Identification (BIMI). If the end user has a BIMI-aware email client, it will check for BIMI DNS records and use them to display a digitally signed corporate logo next to the email. BIMI is built on DMARC, so it cryptographically verified.
Deploying BIMI starts with “register an expensive trademark” and “purchase an artificially expensive certificate,” so these two paragraphs are this book’s only coverage of the topic.
Mail Transfer Agent Strict Transport Security (MTA-STS)
Originally, email was transmitted unencrypted. Anyone with a properly placed packet sniffer could eavesdrop on all email transactions. We eventually added encryption to user authentication, securing usernames and passwords, but using TLS between MTAs was optional. Most mail operators who deployed TLS considered large-scale capture programs like Carnivore their primary risk, so self-signed certificates became acceptable even though they do not prevent other attacks.
Mail Transfer Agent Strict Transport Security (MTA-STS) is a declaration that your mail server only accepts SMTP over TLS. Your mail server must offer a policy statement on a web site. MTA-STS is activated via a DNS record, using the _mta_sts identifier.
TLS Reporting (TLS-RPT)
Every sysadmin has seen email bounce messages, declaring that the recipient has no account or that a server is misconfigured or that the receiving MTA is on an unscheduled vacation. These messages go between MTAs via SMTP.
If your MTA uses MTA-STS to declare it only accepts connections over TLS, and it has a TLS problem, those SMTP messages will not go through. Your MTA will continue merrily rejecting connections, leaving you ignorant of the problems. TLS reporting, or TLS-RPT, is how you MTAs exchange reports of TLS problems. If you deploy MTA-STS, you also need TLS reporting.
Like everything else in email, TLS-RPT information is distributed via DNS.
DNS-based Authentication of Named Entities (DANE)
One of the big objections to common X.509 certificates is the existence of Certificate Authorities (CA). While certificate prices have plunged, a compromised CA could still do immense damage to the confidentiality and integrity of its users.
DNS-based Authentication of Named Entities (DANE) is a way for organizations to publish X.509 certificate fingerprints in DNS, as documented in RFC 7671. It allows a domain owner to authenticate both self-signed and CA certificates. DANE is one of those contentious topics in security circles, and bringing it up at a technical conference is a great way to start the kind of argument that makes bystanders call law enforcement. The arguments about certificate authorities will not be settled by myself or (probably) anyone reading this book. Different people have different concerns about different risks and pain points, that’s all.[5] If someone has gone to the trouble of publishing DANE information for their mail servers, we should respect them.
DANE records are published in TLSA DNS records.
Which Do You Need?
These protocols were developed years apart. Some of their functions overlap. Which do you need, and which can you ignore?
Many people deployed the protocols available when they set up their server and never went circled back to add new ones. You’ll see servers with only SPF, or only SPF and DKIM. Which you need isn’t just a question of what you want to offer the world. It’s about what other servers can validate. If an MTA can validate only SPF, but you offer only DMARC, the recipient might consider your mails suspicious.
To maximize deliverability, and to be aware of any problems, implement SPF, DKIM, and DMARC for your outbound mail. To minimize junk mail, verify any and all of SPF, DKIM, and DMARC for your inbound mail.
Do you need MTA-TLS and TLS-RPT? That depends on your threat model. Maybe you don’t care incoming connections are encrypted or not. It’s fairly new, so perhaps your MTA doesn’t yet support it. If you’re deploying a new mail system or taking the time to polish your existing system, however, you might as well experiment with it. Use a very low timeout when testing, so remote servers won’t cache your mistakes for months or years.
What’s In This Book?
Chapter 0 is this introduction.
<<<add in as I write more>>>
[1] Hopefully you stop before you install WordPress and join the dark side. Unlike me.
[2] While I have written books on TLS, DNSSEC, and OpenPGP, as well as the infamous Networknomicon, I retain enough sanity and self-respect to not write a core DNS book.
[3] Unless your network connection involves phrases like “UDP over carrier pachyderm,” “we get phone service every third Sunday,” or “Callisto Colony One.” Then do what you gotta.
[4] If the forger’s address is in my SPF record, I am having a very bad day.
[5] Rule of System Administration #3 applies here. “Nothing is secure. Everything is terrible.”
My next tech book will be “Run Your Own Mail Server,” part of the IT Mastery series. I would like a snappier title. “Lay Down All Hope, You That Go In By Me” is accurate to the spirit, but doesn’t inform about the content. Now that the Devotion & Corrosion Kickstarter is over, I could theoretically open sponsorships for the new book. I’m not doing it yet.
I work to be honest in my books. I can’t yet express this book’s truth in a short pithy way suitable for a catalog.
Email is a huge topic. Postfix, exim or Exchange? Dovecot, Cyrus, or Courier? Sendmail or syphilis? What exactly is this book about, anyway? I’m using a couple programs for my reference implementation, but this is not exactly a book about system administration. It is about citizenship and society. A novice sysadmin will not be able to use this book without reading a bunch of other books first. This is mostly about how the system hangs together, and about the less well-known services that help email happen. SPF and DKIM, DMARC, MTA-STS, and TLS-RPT. How not to warm up your IP address. Defeating Google. IPv4 or IPv6?
The easiest way to communicate this is by posting Chapter 0 before I open sponsorships.
Which means that the chapter must be vaguely readable. It will be riddled with errors, but it will discuss what’s further in the book.
A handful of you don’t care what the book is about. You’ll sponsor it anyway. You’d sponsor a book about ls(1). You folks are the best, and I appreciate your eagerness to give me money for no good reason. Most folks are not so enraptured, however, and want to know something about what I’m writing.
Maybe next week? The week after? I write nonfiction in order but not linearly, so it’s taking longer than I hoped.
But book is happening. Eddie Sharam’s cover art is glorious, and it’s an actual painting again. It might be a Kickstarter reward, though I think he’d make more if he put it up for direct auction. We’ll find out.
I do know that email is overwhelmingly dominated by a handful of big companies. An Email Empire, if you will. Those of us who run our own email are a ragtag band of rebels struggling against the Empire, constantly hoping not to be crushed as we carve out our little space. Gmail has altered the email agreement, and we pray they do not alter it further. All we need is a Bigfoot with an energy-crossbow-thingamajig…
Folks like getting books from me personally, so normally I bring a few copies of each old title and about ten copies of each book released since the previous con. I’ve released eighteen books since the last meatspace BSDCan, plus innumerable chapbooks and collections and anthologies and who knows what all. 18 titles, ten of each? I am fifty-six years old and am not lugging one hundred eighty books into a con knowing full well that most won’t sell and I must lug them back to the car and unload them at home and let them collect dust. Nope. Ain’t gonna. Can’t make me.
Plus, there’s this.
One copy of every edition of everything I’ve written, 2023 February
I have written too many books to drag “a couple copies” of everything anywhere, let alone to BSDCan. Also, I haven’t done cons for years and have methodically reduced inventory. Other than the copies on this brag shelf I own very few copies of anything. I can buy more from the printer, but I’m not going to buy 180 dust collectors books.
The obvious answer is to let y’all preorder for delivery at BSDCan. I can get exactly what I need, plus a couple extras here and there.
To order a book for delivery at BSDCan, check my meatspace catalog for titles of interest. All prices are USD. Send your list to mwl at mwl dot io on or before before 1 April 2023, specifying paperback or hardcover when there’s a choice. Use the subject “BSDCan Preorder.” I will confirm the total price.
Payment? If you want one or two books, you can pay me cash at BSDCan. Three or more books, I need payment before 3 April. (Otherwise, some jerk who isn’t even attending BSDCan will order thousands of dollars worth of crap just to screw me and call it a joke.) I’ll use my tip jar for prepayments because it takes credit cards or paypal, or you can paypal to accounts at tiltedwindmillpress dot com.
I will place my order on 2 April. I must have prepayments by then.
I will deliver at BSDCan, 19-20 May 2023. Folks who haven’t prepaid should pick up on the 19th (or tell me why not), otherwise I’ll assume you’ve changed your mind and try to get rid of them on the 20th.
Will I bring books for people who don’t preorder? Yep. A few. Not many. You’re better off preordering. Price increases on print books are coming this month, but I will honor these prices for these preorders.
While the meatspace catalog includes everything I can get, here’s what’s come out since the last BSDCan.
Tech books since BSDCan 2019
Sudo Mastery, 2nd edition
SNMP Mastery
The Networknomicon
TLS Mastery (Beastie Edition, Tux Edition, and combined hardcover available, be specific, although I’m sure you fine folks will all want Beastie or the hardcover)
DNSSEC Mastery, 2nd edition
OpenBSD Mastery: Filesystems
Other Nonfiction since BSDCan 2019
Cash Flow for Creators
Only Footnotes
Domesticate Your Badgers
Letters from ed(1): The FreeBSD Journal Letters column, years 1-3
Fiction Since BSDCan 2019
Drinking Heavy Water (Montague Portal)
Terrapin Sky Tango (Beaks #2)
Aidan Redding Against the Universes (Montague Portal all-inclusive omnibus)
$ git sync murder ($ git commit murder #2)
Prohibition Orcs (collection)
Frozen Talons (Prohibition Orcs #2)
Devotion and Corrosion (collection)
Huh. I feel like I’ve been slow during the pandemic, but this is a modestly respectable number of books. The years know things that the hours and days cannot.
Anyway. Check the meatspace catalog for what you want. Send me a list. Specify paperback or hardcover. I will verify price. Three or more, pay me beforehand. Pick up at BSDCan.
Otherwise, take your chances that I’ll bother to bring the book you want and won’t sell it before you can grab it.
I have my ebookshop at tiltedwindmillpress.com, and it’s a great defense against online retailers deciding that they don’t want my business. But sometimes it comes back to bite me.
I don’t want your personal information, unless things go wrong. But when it goes wrong, I have no way to contact you.
I got an email today from antonio@z*****.com, saying they hadn’t gotten the BookFunnel email for their purchase. They had received the receipt from me. It happens. I replied, and my maillog spat out:
In between them getting the receipts and BookFunnel sending the email, their mail server broke.
I have no other information by which to reach this reader. They did provide an address, which is nice. I could send a postcard. To another country. Assuming that address is even correct, which is dubious given the privacy habits of y’all. (Yes, I collect addresses, this is why.)
So, what do I do here?
Nothing.
Presumably, the customer will notice that they aren’t getting any email and reach out again.
Although I do see that the customer has four MXs, and they’re all the same priority. It might be one of them is misconfigured. I’ll send another message, see what happens. Or, maybe the customer will see this blog or my fedi post and see that they have email trouble.
If I demanded phone numbers for ebook purchase I could text them, but that would require I collect phone numbers and I want to not have that information.
I am pleased to see BSDCan returning to meatspace. Yes, the pandemic is ongoing, and I don’t blame folks who decide not to attend. The main reason I chose to attend is that the concom (which I’m a member of) has chosen to enforce a stringent mask policy. Yes, I know you have the right to not wear a mask in public. BSDCan is a private event for a community, however, and communities have a responsibility to protect their weakest members. (People who think that your rights outweigh your responsibilities, I will delete your comments.)
I hope to see many of you at the con, if not my tutorial. It will be good to see many old friends. Well, at least their eyes. Faces in one of the many fine outdoor dining establishments Byward Market offers.
Somehow, I’ve written 28 “We Get Letters” columns for the FreeBSD Journal. The latest is out. I’m amazed that they haven’t given me the boot yet, especially as I’m attempting to channel Michael Bywater’s brilliant “Bargepole” column from the unforgivably murdered weekly “Punch.” At best I’ve achieved a tenth of Bargepole’s vitriolic geezerliciousness, but seeing as Bargepole contained enough vitriol to kill a beluga whale I expected them to ban me years ago.
You can get all the columns for free by trawling through the old issues, but you could also grab the collection and save yourself three years of trouble.
Me in 2014: “Okay, you’ve published post-apocalyptic sci-fi novels. Stay in that genre, don’t dilute your brand.”
Me in 2015: “Okay, you’ve published post-apocalyptic sci-fi novels and bright future sci-fi. That’s not bad.”
Me in 2016: “Okay, you’ve published post-apocalyptic sci-fi novels, bright future sci-fi, and crime thrillers. They’re all exciting stuff, but focus on the types of things you’ve published. You aren’t Iain M Banks, even if a W looks like an upside-down M.”
Comparing how two different books sell is foolish. Books are not fungible. Comparing how two different marketing campaigns do is likewise foolish. Neither marketing, nor readers, nor economic conditions are interchangeable.
But bear with me for a moment while I do it anyway.
The Kickstarter campaign for the Prohibition Orcs duology brought in about $11,000. Which, while not Brandon Sanderson’s millions, was way cool.
I intended to use the Devotion and Corrosion campaign to compare “Kickstarter with Twitter” to “Kickstarter without Twitter.” That seemed sensible, right? The videos are comparable, the campaigns are comparable. The story for the orcs is stronger, because “Orcs in Prohibition” is a solid hook, but still, it’d do something, right?
But then Kickstarter did not give D&C the “Projects We Love” button, which meant that they won’t promote it for me. My attempted comparison fails. I suspect this is because D&C has less Big Idea and more gentle philosophical ambiance. We don’t like to think, we want orcish face-punching.
The campaign funded anyway, which is great. But I can’t do even my lame, heavily-caveated comparison. Facebook spreads my posts to a few of people I know and a couple of the folks who follow my fan page. While the Fediverse shares my posts more broadly, it’s definitely a less commercial space.
I had hoped D&C would hit $4,000. My marketing has saturated the people I know, however, and now I’m hoping it breaks $3000.
Disappointing? Not really. Thinking about it in pure financial terms is wrong.
D&C is doing “well enough.” A bunch of people, wonderful people, will get ebooks and paperbacks. A handful of folks (those with disposable income as well as exquisite taste) will get fancy leather-cased books covered in rats and brains and knives. I will cover expenses and most of a mortgage payment.
And I can be confident that my readers who follow me on the fediverse or my mailing lists or my blog or even on Facebook will at least consider at my next project, no matter what any CEO or billionaire has to say about it. Eventually, a fiction book from this no-name nobody will pretty reliably cover a mortgage payment.
I have what Twitter’s owner can never have.
Enough.
But if you want that fancy leather-cased book, you better grab one quick. Once the campaign is fulfilled, I’m not getting Studio 42 to make any more. In fact, let me post that picture, because it really is spectacular.
Comparing one book to another, or one Kickstarter to another, is foolish, but I can guarantee that you won’t find that many rats on any other leather-cased book.
