bsd – Page 16 – Michael W Lucas

November 1, 2013

FreeBSD Jails and ntpd

I’ve written elsewhere how daemons running on jail servers (the main host, not the imprisoned machines) should listen only on a single address. They shouldn’t bind to all addresses on the machine.

Your average empty FreeBSD install has two problem children: syslogd and ntpd. Adding syslogd_flags="-ss" to /etc/rc.conf handles the first. But FreeBSD’s included ntpd binds to port 123 on all addresses on the machine.

You can run jails while running ntpd. The jail won’t crash in flames. But the jail code expects the jail to have exclusive access to the jail address. This could well come back to bite you later. Besides, it lacks elegance.

Enter openntpd. Openntpd can synch your host clock without binding to any ports. Install it from packages:

# pkg install openntpd

The file /usr/local/etc/ntpd.conf lets you set the preferred server(s) and, if needed, a bind address. This machine is in private address space, so I have to point it at my local time server.

server time.michaelwlucas.com

Now enable openntpd in /etc/rc.conf, and disable the system default ntpd if it’s running.

openntpd_enable="YES"

Run ntpdate to fix the time, then start openntpd.

# ntpdate time.michaelwlucas.com 1 Nov 15:03:22 ntpdate[53689]: adjust time server 192.0.2.130 offset -4.001088 sec # service openntpd start Starting openntpd.

The clock is now correct — or, rather, if the clock is wrong, all the servers will be wrong together. And the various jails each has sole access to their own IP addresses.

October 29, 2013

“Sudo Mastery” ebook widely available, and acknowledgements

At long last, Sudo Mastery is now available in ebook form on most platforms.

You can get it at my bookstore or Amazon.

It’s also available at Smashwords, but Smashwords doesn’t support footnotes. They do support a workaround that puts all footnotes together at the end of a chapter or the end of the book, but it’ll take some work on my part to make that happen.

It’s not at Barnes & Noble yet, because their new Nook Press application completely mangled the book’s formatting. As I sell an average of one book a month through B&N, I’m seriously considering not having the book there.

Print will come some time in November.

I appreciate all the people who helped me write this book. So, in that spirit, here are the acknowledgements.

I want to thank the folks who reviewed the manuscript for Sudo Mastery before publication: Bryan Irvine, JR Aquino, Hugh Brown, and Avigdor Finkelstein. Special thanks are due to Todd Miller, the current primary developer of sudo, who was very patient and helpful when answering my daft questions.

While I appreciate my technical reviewers, no errors in this book are their fault. All errors are my responsibility. Mine, do you hear me? You reviewers want blame for errors? Go make your own.

XKCD fans should note that the author does not particularly enjoy sandwiches. However, Miod Vallat, currently exiled to France, would really like a sandwich with nice fresh bread, really good mustard, and low-carb ground glass and rusty nails. And Bryan Irvine would like a rueben.

This book was written while listening obsessively to Assemblage 23.

Now, to finish writing my big 2013 fiction project before the end of the year…

October 7, 2013October 7, 2013

EuroBSDCon, and Sudo Mastery

How’s that for diverse topics in one post?

I just got back from EuroBSDCon 2013 on Malta. The EuroBSDCon Foundation and Andre Opperman did a great job with the conference, and presented one of the best sets of talks and keynotes and related programs I’ve seen in years. It’s motivated me to try to improve BSDCan, but I’ll babble about that in another post.

I followed EuroBSDCon with a few days in Paris, to talk to other authors, network, and figure out some “business of writing” stuff. Plus see the Eiffel Tower and the catacombs, of course.

Now that I’m home, I’m diving into the technical reviews of Sudo Mastery.

Normally when I have a book out for tech review, I post a variety of reminders during the time people should be reviewing. “Two weeks to get comments back to me!” “One week!” “Six hours and three minutes!” I didn’t do that this time, instead focusing on things like distributing blacklists via BGP and automated deployment of FreeBSD and Bhyve and relayd and and and and…

In a weird coincidence, I haven’t received as many tech reviews as I usually do.

Why do people nag? Because it works.

If you’re one of the folks who volunteered to review the manuscript, you have a couple days left to send me comments. I would really like to get the book to the copyeditor by next Monday.

August 16, 2013

Command-Line FreeBSD Configuration: sysrc

The traditional BSD standard of “edit /etc/rc.conf” isn’t sustainable across large numbers of machines. If you must change dozens of servers you want a reliable way to alter the system without either manually editing every configuration file or some sed/awk hackery. (Running a sed/awk script to edit rc.conf on every server I own makes me nervous. I don’t do nervous these days.) FreeBSD 9.2 and later includes sysrc, a program to consistently and safely alter /etc/rc.conf and friends from the command line. (On older versions of FreeBSD, sysrc is available in ports.) I find sysrc very useful for Ansible-managed farms, but it should work just as well with Puppet or Chef or any configuration management system.

Start using sysrc by asking it what it knows about the system configuration.

$ sysrc -a defaultrouter: 192.0.2.129 dumpdev: AUTO hostname: mwltest3 ifconfig_em0: inet 192.0.2.160 netmask 255.255.255.128 keymap: us.dvorak.kbd sshd_enable: YES

Oddly enough, this is exactly what’s in my rc.conf, minus all the comments and such.

I must enable ntpd(8) on this machine. Here, sysrc looks an awful lot like sysctl.

# sysrc ntpd_enable=YES ntpd_enable: NO -> YES

Note that this doesn’t actually start ntpd, it merely enables it in the configuration. You must run /etc/rc.d/ntpd start or service ntpd start to start the daemon.

To disable a daemon, do the same thing in reverse.

# sysrc ntpd_enable=NO ntpd_enable: YES -> NO

One potential problem with sysrc is that it’s a tool for editing /etc/rc.conf, not for configuring FreeBSD. It has no idea what legitimate values are. This means that if you make a typo, it propagates to rc.conf.

# sysrc ntpd_enable=YSE ntpd_enable: NO -> YSE

Here sysrc works as advertised, but ntpd won’t. And you can arbitrarily enable nonexistent services.

# sysrc gerbil_enable=YES gerbil_enable: -> YES

I do not have gerbils installed. But if I ever do install them, the gerbil wheel will start spinning without any further intervention.

I suspect that one day sysrc will grow a service integrity checker, but it solves my immediate needs.

Experienced FreeBSD administrators who run a small number of servers don’t need sysrc, but those of us with farms will find it invaluable. My thanks to Devin Teske for shepherding this into base.

July 29, 2013July 29, 2013

FreeBSD pkgng vs custom ports

My Web server runs FreeBSD. Some security updates prompted me to upgrade my installed packages. Running this server entirely with packages isn’t possible, so I installed PHP 5 from ports. This machine uses pkgng.

When I ran pkg upgrade, however, my web site stopped working. The server itself started just fine, and it did quite well serving downloadable PHP code to clients. The problem was pretty obvious:

I originally compiled PHP 5 from ports so to get the Apache PHP module. Running pkg upgrade replaced my custom-built PHP with one from packages, so Apache no longer had a PHP module.

So: I want to upgrade from packages, but not upgrade PHP automatically. pkgng has a tool for this, pkg lock. You must give pkg lock a package name and confirm that you want to update this package.

# pkg lock php5 php5-5.4.17: lock this package? [y/N]: y Locking php5-5.4.17 #

If you give a nonexistent package name, pkg exits silently. Which is kinder than calling you an idiot, I suppose.

# pkg lock php-5 #

Personally, I’d rather be told I’m an idiot (aka “no such package”), because when I’m tired I might interpret this as “package locked, everything is good.” But whatever.

Before uninstalling a locked port, either via pkg or ports, you must unlock the package.

To upgrade this server now, I do the following:

# portsnap fetch update # pkg upgrade # cd /usr/ports/lang/php5 && make # pkg unlock php5 # make deinstall && make reinstall # pkg lock php5

Done!

I’m told by people who should know that eventually pkgng will let me do this entirely with packages, but for now, this will do nicely.

From my experience, upgrading with pkgng is MUCH nicer than upgrading with pkg_add. All of the issues that drove me away from binary package upgrades have disappeared.

I should also note, however, that the BSDs will have the ports tree for the forseeable future. We need it. Some popular ports, such as nginx, have 77,371,252,455,336,267,181,195,264 possible combinations. The packaging team is not going to build 2^86 nginx packages. But you can build whichever exact version you need.

It seems that pkgng is actually taking FreeBSD binary packaging back towards being useful. Once there’s an official public repository of pkgng packages, you really should try it out.

(Update: Hat tip to Allan Jude for pointing out that 2^86 is an impressively big number when you multiply it out.)

July 22, 2013

convert FreeBSD to pkgng with Ansible

Ansible includes a module to manage FreeBSD packages, if you’re using the forthcoming pkgng packaging system. The Ansible module isn’t complete yet, but as Ansible is moving really quickly, I’m pretty confident their FreeBSD support will grow additional knobs. As pkgng is increasingly close to production, and the PC-BSD folks have generously offered their 64-bit pkgng repository available to the public, this seems like a good time to make the move.

But I’m not about to make this change manually. Bootstrapping pkgng isn’t difficult, but I have a great big heap of FreeBSD VMs and I have other things I’d like to accomplish this month. Therefore, I’m bootstrapping my ability to manage FreeBSD packages via Ansible, with Ansible.

Before starting, you need an Ansible server and a pkgng repo.

All of my FreeBSD servers run 9.1, updated via freebsd-update. If you come across this article years later, adjust accordingly.

My Ansible server runs OpenBSD, and the OpenBSD ansible package has problems managing anything other than OpenBSD. I generically recommend running Ansible out of git.

You also need a pkgng repository. The official repository is in closed testing, but many FreeBSD developers are using it successfully. PC-BSD has made their 64-bit repository available to all FreeBSD users. And many people have built their own repository. Thanks to my awesome Twitter ~~stalkers~~ ~~minions~~ followers, I have access to more than one private repository. This example assumes you’re using the 64-bit-only PC-BSD repository.

Configure the pkgng repo in pkg.site. I keep my FreeBSD configuration files in /home/ansible/freebsd/etc/, so I make a /home/ansible/freebsd/etc/pkg.site that contains only:

packagesite: http://pkg.cdn.pcbsd.org/9.1-RELEASE/amd64 PUBKEY: /usr/local/etc/pkg-pubkey.cert PKG_CACHEDIR: /usr/local/tmp

I also need the current PC-BSD public key, saved as pkg-pubkey.cert

With these two files and an Ansible install, we’re ready to deploy on the Ansible group freebsd-test. Here’s the runbook.

---
- hosts: freebsd-test
  user: ansible
  sudo: yes

  tasks:
  - name: install pkg tools
    action: command  pkg_add -r pkg
#do you need a proxy? Put it here
#    environment:
#      ftp_proxy: http://proxy.michaelwlucas.com:8080

  - name: edit /etc/make.conf
    action: shell echo "WITH_PKGNG=YES" >> /etc/make.conf

  - name: convert package database
    action: shell pkg2ng

#I have typed pkg_add for 18 years, and my fingers no longer listen to
#my brain. Disable pkg_* commands for safety
  - name: disable pkg_ commands
    action: shell chmod -x /usr/sbin/pkg_*

  - name: install pkg.conf
    action: copy src=/home/ansible/freebsd/etc/pkg.conf
      dest=/usr/local/etc/pkg.conf owner=root group=wheel mode=0644

#skip this if you're using a non-PCBSD repo
  - name: install pc-bsd pgp key
    action: copy src=/home/ansible/freebsd/etc/pkg-pubkey.cert
      dest=/usr/local/etc/pkg-pubkey.cert owner=root group=wheel mode=0644

#ansible pkg does not have upgrade command yet
#use shell to trigger upgrade
#pkgng package in pkg-old is always out of date, upgrade it
  - name: upgrade pkg pkg
    action: command pkg upgrade -qy 
#do you need a proxy? Put it here
#    environment:
#      ftp_proxy: http://proxy.michaelwlucas.com:8080

This takes a while to run.

Before deploying, test. Test again. And run your conversion in batches, so that you don’t scramble several hundred virtual machines simultaneously. Because that would really suck. Fortunately, by changing the group at the top of the playbook or specifying a new inventory file, you can batch these changes easily.

July 12, 2013

I sound like a real dork: BSDTalk 228

Want to hear my voice? Interested in Absolute OpenBSD or DNSSEC Mastery?

Go check out BSDTalk 228, starring yours truly.

July 10, 2013

“Absolute OpenBSD” auction photographs

I promised to post photographs of the signed copy of Absolute OpenBSD 2nd Edition that went to Bill Allaire.

Here they are. In full size, so that Bill can later use them to authenticate his copy, in the extraordinarily unlikely event that he desires to do so. (And also so that perhaps the bandwidth utilized might finally exceed that generated by my fame in the gay porn world.)

All the developers signed the first page:

People immediately checked the index for their favorite features. Leave it to Bob Beck to find my extraordinarily subtle joke on BUFCACHEPERCENT, which I assure you I completely deliberately chose to do and was not at all an actual error.

Some developers weren’t terribly confident of their English skills. But that’s okay.

Sadly, there were some problems with the book. People were kind enough to point them out.

Many pages didn’t get marked up, but a few got extra attention. We now know exactly how Theo feels about file flags.

Finally, Theo was kind enough to add a footnote. He didn’t mark it from the main body of the text, however. Programmers might have mad skillz, but footnotes are much harder than they look.

As an aside: I considered getting the developers to sign my personal copy of the book while I was there. But that would reduce the uniqueness of this artifact, dang it. So Bill gets the awesome copy and the rest of us get pix.

June 5, 2013

Visiting an OpenBSD hackathon.

I took Bill Allaire’s copy of Absolute OpenBSD to Toronto to get it signed. If you wonder what that’s like, check out my article over on undeadly.org.

Why no tech posts from me for a while now? I’m moving a bunch of virtual machines from ESX to SolusVM. As some of these VMs are several years old, I’m taking the opportunity to make new OS installs and get them into my Ansible setup. It’s a lot of work, but it’s not blog-worthy.

When I do something interesting, you’ll be the first to know. Well, no, maybe the sixth or seventh. But still, pretty early on.

May 21, 2013

FreeBSD-update vs bind99-base

My master nameserver runs BIND 9.9, so I can do DNSSEC easily. I’ve installed from ports, but used the REPLACE_BASE option so that it overwrites the BIND 9.8.3 install included in the base system. That way I don’t have to worry about having multiple versions of the same command on different systems.

I patch this system via freebsd-update. After applying the latest security patches, I got the following email:

The following files will be updated as part of updating to 9.1-RELEASE-p3: /usr/bin/dig /usr/bin/host /usr/bin/nslookup /usr/bin/nsupdate /usr/sbin/ddns-confgen /usr/sbin/dnssec-dsfromkey /usr/sbin/dnssec-keyfromlabel /usr/sbin/dnssec-keygen /usr/sbin/dnssec-revoke /usr/sbin/dnssec-settime /usr/sbin/dnssec-signzone /usr/sbin/lwresd /usr/sbin/named /usr/sbin/named-checkconf /usr/sbin/named-checkzone /usr/sbin/named-compilezone /usr/sbin/named-journalprint /usr/sbin/rndc-confgen

I don’t want freebsd-update to patch these files. I also don’t want to get an email every day telling me that I need to patch them. I know I don’t need to patch them.

The solution? Tell freebsd-update to ignore these files with the IgnorePaths directive in /etc/freebsd-update.conf. I copied the list of files from the email and added IgnorePaths before them.

... IgnorePaths /usr/bin/dig IgnorePaths /usr/bin/host IgnorePaths /usr/bin/nslookup IgnorePaths /usr/bin/nsupdate IgnorePaths /usr/sbin/ddns-confgen IgnorePaths /usr/sbin/dnssec-dsfromkey IgnorePaths /usr/sbin/dnssec-keyfromlabel IgnorePaths /usr/sbin/dnssec-keygen IgnorePaths /usr/sbin/dnssec-revoke IgnorePaths /usr/sbin/dnssec-settime IgnorePaths /usr/sbin/dnssec-signzone IgnorePaths /usr/sbin/lwresd IgnorePaths /usr/sbin/named IgnorePaths /usr/sbin/named-checkconf IgnorePaths /usr/sbin/named-checkzone IgnorePaths /usr/sbin/named-compilezone IgnorePaths /usr/sbin/named-journalprint IgnorePaths /usr/sbin/rndc-confgen ...

The complication here is that I must watch out for BIND security advisories, rather than just trusting in the update process. But that’s normal.