Blog

After years of only needing central auth for Unix-like systems, I need to integrate Windows clients into my auth mix. Rather than munging my current OpenLDAP directory to contain Windows information, I elected to migrate to Samba 4. Samba 4 can act as a Windows domain controller and also exposes an LDAP interface for Unix clients.

I assume that you’ve read the FreeBSD and Samba documentation on LDAP auth. This article is meant as a bridge between the two information sets.

Most of the tutorials out there cover using LDAP and Kerberos with Samba. Kerberos requires that all hosts be in a single domain. My employer manages hosts within a variety of domains and business units, so Kerberos is a no-go. I need to use pure LDAP authentication.

I configured my samba4 domain and ensured that Windows clients could join the domain and that the general Microsoft-esque features worked, such as failover to a backup domain controller. Once it appeared that everything worked, I set up a couple of OpenLDAP proxies exactly by the Samba documentation. (My domain controllers are in private address space, and I’m not willing to expose them to the larger network.)

Then I created an account for services to bind with to perform basic queries. My binding account is called unixstuff. It’s not a member of any AD groups.

Then I can configure OpenLDAP on the client. Install the various LDAP utilities as in the FreeBSD documentation.

Configure the OpenLDAP tools to query your directory first. I use a private CA, so I set assorted TLS options.

BASE dc=internal,dc=mwlucas,dc=org
URI ldap://snarky.mwlucas.org
ssl start tls
tls_cacert /usr/local/etc/ssl/mwlucas.crt

At this point I should be able to use ldapsearch on the Samba directory.

# ldapsearch -WxD "cn=unixstuff,cn=users,dc=internal,dc=mwlucas,dc=org"
Enter LDAP Password:
...

If you enter the correct password, the directory should spill its guts.

Once that works, we can configure LDAP authentication in /usr/local/etc/ldap.conf.

host snarky.mwlucas.org
base dc=internal,dc=mwlucas,dc=org
ldap_version 3
binddn cn=unixstuff,cn=users,dc=internal,dc=mwlucas,dc=org
bindpw WhyHardCodePasswords

pam_filter objectclass=posixAccount
pam_login_attribute uid
pam_member_attribute member
pam_lookup_policy yes
nss_base_passwd cn=users,dc=internal,dc=mwlucas,dc=org
nss_base_group cn=users,dc=internal,dc=mwlucas,dc=org
pam_groupdn cn=sysadmins,cn=users,dc=internal,dc=mwlucas,dc=org

# Services for UNIX 3.5 mappings
nss_map_attribute homeDirectory unixHomeDirectory
nss_map_objectclass posixAccount User
nss_map_objectclass shadowAccount User
nss_map_attribute uid msSFU30Name
nss_map_attribute uniqueMember msSFU30PosixMember
nss_map_attribute userPassword msSFU30Password
nss_map_objectclass posixGroup Group
pam_filter objectclass=User
pam_password ad

With this set, and LDAP activated in /etc/nsswitch.conf, I can now “getent passwd” and “getent group” and get responses from the directory.

That still leaves PAM. I’m no PAM expert, but I’ve hacked together something that permits LDAP access, and falls back to the local password file when the LDAP servers are unavailable. Here’s my /etc/pam.d/system:


auth sufficient /usr/local/lib/pam_ldap.so
auth required pam_unix.so no_warn try_first_pass nullok

account required pam_login_access.so
account sufficient /usr/local/lib/pam_ldap.so
account required pam_unix.so

session required /usr/local/lib/pam_ldap.so
session required pam_lastlog.so no_fail

password required pam_unix.so no_warn try_first_pass

This gets you basic access. Realistically, though, users want a home directory. Install pam_mkhomedir and enable it on a per-service basis.

auth sufficient /usr/local/lib/pam_ldap.so no_warn
auth required pam_unix.so no_warn try_first_pass

account required pam_nologin.so
account required pam_login_access.so
account required pam_unix.so
account required /usr/local/lib/pam_ldap.so no_warn ignore_authinfo_unavail ignore_unknown_user

session required pam_permit.so
session required /usr/local/lib/pam_mkhomedir.so

password required pam_unix.so no_warn try_first_pass

With this set, users in the “sysadmins” group in AD have shell access to the servers. So far, this works well. But if you’re a Samba, LDAP, or PAM expert and see a problem, please let me know in the comments.

I’m setting up a new FreeBSD web server. As 10.0 is just around the corner, I installed 10.0-BETA2. BETA4 is out, so it’s time to upgrade.

# freebsd-update -r 10.0-BETA4 upgrade
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
...

That all looks good. Then I installed the update

# freebsd-update install
...

And the install hung. Overnight. Why?

Turns out that there are multiple errata on freebsd-update. While you can upgrade to BETA2-p2, you can’t jump directly to the next version.

So you need to do:

# freebsd-update fetch
# freebsd-update install
# reboot
# freebsd-update -r 10.0-BETA4 upgrade
# freebsd-update install

When freebsd-update appears to hang, use ^T to print the current status.

# freebsd-update install
Installing updates...
^T
load: 0.17 cmd: install 2796 [nanslp] 156.82r 0.01u 0.02s 0% 6220k

Here we see that the install process, PID 2796, is running. Wait a few minutes and try it again.

^T
load: 0.16 cmd: install 2803 [nanslp] 26.09r 0.01u 0.01s 0% 6160k

There’s been no visible output, and install is still running, but it’s now PID 2803. It’s a new install process. The update is proceeding, if not as quickly as you’d like.

So, be patient and let the machine do its work.

I have an old mail server running Postfix and courier-imap. We want to split our customers off onto their old server, preferably something with a pretty pointy-clicky interface so that they can manage their own accounts. (Yes, people do still buy email service these days.)

The old server runs FreeBSD, postfix, and courier-imap. The new server runs FreeBSD 9.2 with DirectAdmin. DirectAdmin is a canned package that puts a customer-friendly front end on a whole bunch of standard Unix software. (Why FreeBSD? ZFS. When my customers fill up the hard disk, I can just “zfs send” the whole machine to bigger disk.)

The good news is, both my old server and the new one use Maildir. The bad news is, they’re arranged slightly differently and have somewhat different file formats. Tools exist to do 80% of the migration for you, but my setup has a few edges.

I suspect that nobody else has my exact legacy setup, and my readers don’t care about this level of detail. I also suspect that the migration project will last weeks and I will completely forget how I did it in between individual customer migrations, so I’m writing this anyway. But maybe someone else can learn something from it, even if it’s just “man, I would never hire this Lucas character.”

Create your email domain and all the user accounts in DirectAdmin. I don’t know any way to migrate encrypted virtual user passwords from Postfix into DirectAdmin, so I’ll need to create new passwords for the users. That’s not uncommon for a packaged mail server migration, so my users will live with it. This is a tedious point-and-click operation, but point-and-click is the point of DirectAdmin, so give the job to a minion and get it done.

But how to transparently migrate the mailboxes from the Courier server to Dovecot?

On the old server, the mailboxes are in /disk2/mail/vhosts/domainname/account/. On DirectAdmin, the mailboxes are in /home/userid/imap/domain/account/Maildir/. I can tar up the old accounts and drag the whole directory over to the new server, but I need to massage the directories so that they’re arranged properly.

Courier and Dovecot both use files to store mailbox state information. IMAP state is important. Without it, users will re-download every single message. I don’t care about the bandwidth or the disk I/O this will cause, but the users will complain. Complaints lead to meetings, and meetings lead to the Dark Side. There’s a script to convert from one to the other. Dovecot also has good migration documentation, I recommend you read that before doing your own migration.

The file ownership needs correcting. DirectAdmin creates user accounts for each customer (more or less), and that account needs to own the email files for the various domains.

So, what am I willing to do manually?

I will manually tar up the domain’s mail directory, copy it to the destination server, and untar it in /home/userid/imap/domain/. And I’m willing to provide the username that should own the files. I’m sure both of these could be automated, but I don’t have enough accounts to migrate to do this correctly.

I’m not willing to move files around or run migration scripts. Because these are things that human beings mess up.

There’s ways to do this in shell scripts, but I’m using Perl. Because while my Perl makes small children cry, my shell scripts feature in rituals praising Nyarlathotep.

So, the script:

#!/usr/bin/perl
#migrate from old mail server to new
#run from domain directory
#Takes one argument, the username expected to own the files when done.

unless ($ARGV[0]) {
die "\nNeed a username to own files!\n\n";
}

opendir (USERS, ".") || die "Cannot open current directory";

foreach $user (readdir (USERS)) {

next if ($user =~ /^\.{1,2}$/);
print "Converting $user...\n";
mkdir ("$user/Maildir") || die "Cannot make Maildir";
opendir (USERDIR, "$user") || die "Cannot open user directory";
foreach $file (readdir (USERDIR)) {
next if ($file =~ /^\.{1,2}$|Maildir/);
print "Moving $user/$file\n";
rename ("$user/$file","$user/Maildir/$file")
|| die "Cannot move $file";
}
#next user
}
print "Now performing courier-dovecot migration...\n";
system ("courier-dovecot-migrate.pl --to-dovecot --recursive --convert");
print "Fixing ownership...\n";
system ("chown -R $ARGV[0]:mail .");

Is this pretty trivial? Yep. But the most error-prone part of any process is the part I do. The more of the migration the machine does, the fewer screw-ups in the migration.

Last night’s talk on OpenBSD is now live in the mug.org channel.

Part 1
Part 2

UPDATE: All in one.

Among other things, I compare OpenBSD to Richard Stallman and physically assault an audience member. (Brian was a very good spots, and learned an important lesson about volunteering, e.g.: don’t.) We also talk long long time, memory randomization, PF, BSD license versus GPL, Microsoft, and other OpenBSD stuff.

And, of course, the importance of the VAX.

I was researching next week’s OpenBSD talk and thought “You know, I ought to tell the story about VRRP, CARP, and Cisco. That’s a good one, and it illustrates how the OpenBSD community works and thinks.” It’s been ten years, so I decided to do some research to make sure I had my facts straight.

And I came across the Cisco Nexus 1000V manual. This big and mighty Cisco switch… supports CARP.

This is absolutely hilarious. I laughed so much my sides hurt.

Some of you younger folks are probably wondering what the big deal is. Well…

Back in the late 1990s, Cisco came up with the Virtual Router Redundancy Protocol (VRRP), using some of the lessons of their Hot Standby Router Protocol (HSRP). This was a quick-acting router failover protocol. If one router died, a second would notice and automatically take over for it. VRRP isn’t rocket surgery, it’s just that Cisco’s hardware could now support it and the market demanded it. Fair enough.

But then Cisco patented VRRP.

Cisco announced that anyone could implement VRRP, so long as they didn’t sue Cisco over it. Cisco wanted to offer something to the world, and didn’t want it to come back and bite them. Again, fair enough. Perfectly sensible from Cisco’s perspective.

The OpenBSD folks wanted router redundancy, too. And they wanted it in the base system. But Cisco’s licensing terms were a problem.

The modern BSD license boils down to:

1) Keep our copyright notice on this code
2) Don’t sue us if it breaks

There’s nothing in there about “And don’t sue Cisco if something breaks.” Specifically, the code can be used for any purpose, including suing Cisco. Mind you, you’d have a pretty hard time using OpenBSD code to sue Cisco, but the license doesn’t prohibit it.

So, while the VRRP patent terms were fine for Cisco, they weren’t acceptable under the BSD license.

And the OpenBSD devs wanted redundancy.

What to do? Go off and write your own protocol, the Common Address Redundancy Protocol (CARP). Make it different from VRRP. Field-test the protocol, using your legions of willing lackeys — er, devoted userbase. Make CARP not only a usable replacement for VRRP, but inherently better and stronger. Put the protocol under the BSD license, and give the protocol and code away.

This caused something of a kerfuffle at the time. Ugly accusations flew around. “It’s a VRRP knock-off!” “No, it’s a different protocol!” Great big reams of email were written about the whole thing.

The OpenBSD folks applied to IANA for a protocol number. IANA rejected the application, telling them to use VRRP instead. VRRP was assigned protocol 112. So OpenBSD used protocol 112 for CARP. And putting CARP hosts on a network with Cisco VRRP hosts made Cisco routers crash. The Cisco stack wasn’t robust enough to handle strange packets on the network. Cisco updated their hardware to survive seeing a lone CARP packet.

This escalated the kerfuffle into industry news. You’d see articles in all kinds of industry magazines about OpenBSD versus Cisco.

The OpenBSD folks responded by doing a CARP/VRRP-themed 3.5 release, complete with a Monty Python parody (lyrics, MP3).

And in the end of it all… everyone shut up. Other people started implementing CARP. Because it’s a solid, respectable redundancy protocol. You can get CARP from FreeBSD, Linux, Solaris, and a whole bunch of other vendors…

…including Ciso.

I had plans for today, but I’m too busy laughing. And then I need to go watch some Monty Python.