Author: Michael Lucas

Writer. Sysadmin. Owns this site.
Posted on

The Desktop of Doom

I’m sick of scrounging hardware for writing books. I’m sick of waiting for things to compile, managing disk space, and running out of memory. I finally got so sick of it that I decided to invest some serious cash in a research machine, in the hope that I wouldn’t need to hunt hardware piecemeal for the next five to six years.

I solicited hardware advice from my modest horde of Twitter followers. After diving through realms of documentation, I came to realize that I didn’t want to build my own computer. I rarely configure, buy, or even touch hardware these days unless something goes unspeakably wrong, and there’s enough options out there that were I to attempt to do so, I’d waste both money and time. I certainly could figure it out, but that’s not where my strength is and the expertise I would acquire would be largely wasted outside this one task.

I wanted eight hard drives, so that when the time is right I can write about assorted ZFS configurations. I wanted some flash disk on top of that. I wanted enough oomph to run umpteen virtual machines simultaneously. And I didn’t want to spend a whole lot.

After wandering around the Internet, using assorted online system configuration tools to develop a rough idea of price, I contacted iX Systems and bought a system.

I now have a shiny new desktop system that can “portsnap fetch extract” in two minutes. Because SATA3 is pretty sweet. My only problem is that the portable version of cwm that @blooouuup found for me is really slow & clunky when resizing windows, but I’m sure if I whine at the right people someone will fix it for me. (Now to figure out who the right people are.)

For all those who asked me to share my shopping results: here’s a FreeBSD 11 dmesg. Thanks to bhyve, I can now run whatever I need. It lives under my desk, and is surprisingly quiet when I’m not building x11/xorg in 30 minutes or less. With WITNESS enabled.

iX Systems might not advertise desktop systems, but they’ll do a nice one for you. Even if yours isn’t as awesome as mine.


Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #0 r260064: Mon Dec 30 16:28:06 UTC 2013
root@grind.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610
WARNING: WITNESS option enabled, expect reduced performance.
CPU: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (3700.08-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0x306e4 Family = 0x6 Model = 0x3e Stepping = 4
Features=0xbfebfbff
Features2=0x7fbee3ff
AMD Features=0x2c100800
AMD Features2=0x1
Standard Extended Features=0x281
TSC: P-state invariant, performance statistics
real memory = 34368126976 (32776 MB)
avail memory = 33227055104 (31687 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: < >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3
cpu4 (AP): APIC ID: 4
cpu5 (AP): APIC ID: 5
cpu6 (AP): APIC ID: 6
cpu7 (AP): APIC ID: 7
ioapic0 irqs 0-23 on motherboard
ioapic1 irqs 24-47 on motherboard
kbd1 at kbdmux0
random: initialized
acpi0: on motherboard
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
cpu0: on acpi0
cpu1: on acpi0
cpu2: on acpi0
cpu3: on acpi0
cpu4: on acpi0
cpu5: on acpi0
cpu6: on acpi0
cpu7: on acpi0
attimer0: port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
pcib1: irq 27 at device 1.0 on pci0
pci1: on pcib1
pcib2: irq 33 at device 2.0 on pci0
pci2: on pcib2
pcib3: irq 33 at device 2.2 on pci0
pci3: on pcib3
pcib4: irq 43 at device 3.0 on pci0
pci4: on pcib4
pcib5: irq 43 at device 3.1 on pci0
pci5: on pcib5
pcib6: irq 43 at device 3.2 on pci0
pci6: on pcib6
pci0: at device 4.0 (no driver attached)
pci0: at device 4.1 (no driver attached)
pci0: at device 4.2 (no driver attached)
pci0: at device 4.3 (no driver attached)
pci0: at device 4.4 (no driver attached)
pci0: at device 4.5 (no driver attached)
pci0: at device 4.6 (no driver attached)
pci0: at device 4.7 (no driver attached)
pci0: at device 5.0 (no driver attached)
pci0: at device 5.2 (no driver attached)
pcib7: irq 16 at device 17.0 on pci0
pci7: on pcib7
isci0: port 0xe000-0xe0ff mem 0xfa47c000-0xfa47ffff,0xfa000000-0xfa3fffff irq 16 at device 0.0 on pci7
pci0: at device 22.0 (no driver attached)
pci0: at device 22.1 (no driver attached)
ehci0: mem 0xfbb23000-0xfbb233ff irq 16 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
pcib8: irq 17 at device 28.0 on pci0
pci8: on pcib8
pcib9: irq 18 at device 28.6 on pci0
pci9: on pcib9
em0: port 0xd000-0xd01f mem 0xfba00000-0xfba1ffff,0xfba20000-0xfba23fff irq 18 at device 0.0 on pci9
em0: Using MSIX interrupts with 3 vectors
em0: Ethernet address: 00:25:90:db:d5:94
pcib10: irq 19 at device 28.7 on pci0
pci10: on pcib10
em1: port 0xc000-0xc01f mem 0xfb900000-0xfb91ffff,0xfb920000-0xfb923fff irq 19 at device 0.0 on pci10
em1: Using MSIX interrupts with 3 vectors
em1: Ethernet address: 00:25:90:db:d5:95
ehci1: mem 0xfbb22000-0xfbb223ff irq 23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci1
pcib11: at device 30.0 on pci0
pci11: on pcib11
vgapci0: mem 0xf9000000-0xf9ffffff,0xfb800000-0xfb803fff,0xfb000000-0xfb7fffff irq 16 at device 4.0 on pci11
vgapci0: Boot video device
isab0: at device 31.0 on pci0
isa0: on isab0
ahci0: port 0xf050-0xf057,0xf040-0xf043,0xf030-0xf037,0xf020-0xf023,0xf000-0xf01f mem 0xfbb21000-0xfbb217ff irq 18 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: at channel 0 on ahci0
ahcich1: at channel 1 on ahci0
ahcich2: at channel 2 on ahci0
ahcich3: at channel 3 on ahci0
ahcich4: at channel 4 on ahci0
ahcich5: at channel 5 on ahci0
ahciem0: on ahci0
pci0: at device 31.3 (no driver attached)
pci0: at device 31.6 (no driver attached)
pcib12: on acpi0
pci255: on pcib12
pci255: at device 8.0 (no driver attached)
pci255: at device 9.0 (no driver attached)
pci255: at device 10.0 (no driver attached)
pci255: at device 10.1 (no driver attached)
pci255: at device 10.2 (no driver attached)
pci255: at device 10.3 (no driver attached)
pci255: at device 11.0 (no driver attached)
pci255: at device 11.3 (no driver attached)
pci255: at device 12.0 (no driver attached)
pci255: at device 12.1 (no driver attached)
pci255: at device 13.0 (no driver attached)
pci255: at device 13.1 (no driver attached)
pci255: at device 14.0 (no driver attached)
pci255: at device 14.1 (no driver attached)
pci255: at device 15.0 (no driver attached)
pci255: at device 15.1 (no driver attached)
pci255: at device 15.2 (no driver attached)
pci255: at device 15.3 (no driver attached)
pci255: at device 15.4 (no driver attached)
pci255: at device 15.5 (no driver attached)
pci255: at device 16.0 (no driver attached)
pci255: at device 16.1 (no driver attached)
pci255: at device 16.2 (no driver attached)
pci255: at device 16.3 (no driver attached)
pci255: at device 16.4 (no driver attached)
pci255: at device 16.5 (no driver attached)
pci255: at device 16.6 (no driver attached)
pci255: at device 16.7 (no driver attached)
pci255: at device 19.0 (no driver attached)
pci255: at device 19.1 (no driver attached)
pci255: at device 19.4 (no driver attached)
pci255: at device 19.5 (no driver attached)
pci255: at device 22.0 (no driver attached)
pci255: at device 22.1 (no driver attached)
pci255: at device 22.2 (no driver attached)
acpi_button0: on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 10 on acpi0
orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0
sc0: at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
est0: on cpu0
p4tcc0: on cpu0
est1: on cpu1
p4tcc1: on cpu1
est2: on cpu2
p4tcc2: on cpu2
est3: on cpu3
p4tcc3: on cpu3
est4: on cpu4
p4tcc4: on cpu4
est5: on cpu5
p4tcc5: on cpu5
est6: on cpu6
p4tcc6: on cpu6
est7: on cpu7
p4tcc7: on cpu7
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
random: unblocking device.
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: at usbus0
uhub0: on usbus0
ugen1.1: at usbus1
uhub1: on usbus1
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
ses0 at ahciem0 bus 0 scbus7 target 0 lun 0
ses0: SEMB S-E-S 2.00 device
ses0: SEMB SES Device
da2 at isci0 bus 0 scbus0 target 2 lun 0
da2: Fixed Direct Access SCSI-5 device
da2: Serial Number WD-WCAW36477223
da2: 300.000MB/s transfers
da2: Command Queueing enabled
da2: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
da0 at isci0 bus 0 scbus0 target 0 lun 0
da0: Fixed Direct Access SCSI-5 device
da0: Serial Number WD-WCAW36478143
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
da3 at isci0 bus 0 scbus0 target 3 lun 0
da3: Fixed Direct Access SCSI-5 device
da3: Serial Number WD-WCAW36477062
da3: 300.000MB/s transfers
da3: Command Queueing enabled
da3: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
da1 at isci0 bus 0 scbus0 target 1 lun 0
da1: Fixed Direct Access SCSI-5 device
da1: Serial Number WD-WCAW36498185
da1: 300.000MB/s transfers
da1: Command Queueing enabled
da1: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: ATA-8 SATA 3.x device
ada0: Serial Number WD-WCAW36478129
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus2 target 0 lun 0
ada1: ATA-8 SATA 3.x device
ada1: Serial Number WD-WCAW36477352
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus3 target 0 lun 0
ada2: ATA-8 SATA 3.x device
ada2: Serial Number WD-WCAW36477139
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus4 target 0 lun 0
ada3: ATA-8 SATA 3.x device
ada3: Serial Number WD-WCAW36477141
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus5 target 0 lun 0
ada4: ATA-8 SATA 2.x device
ada4: Serial Number 20131111AA300000000E
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 61057MB (125045424 512 byte sectors: 16H 63S/T 16383C)
ada4: Previously was known as ad12
ada5 at ahcich5 bus 0 scbus6 target 0 lun 0
ada5: ATA-8 SATA 2.x device
ada5: Serial Number 20131111AA3000000007
ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada5: Command Queueing enabled
ada5: 61057MB (125045424 512 byte sectors: 16H 63S/T 16383C)
ada5: Previously was known as ad14
ugen0.2: at usbus0
uhub2: on usbus0
ugen1.2: at usbus1
uhub3: on usbus1
Netvsc initializing... done!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #5 Launched!
Timecounter "TSC-low" frequency 1850039224 Hz quality 1000
WARNING: WITNESS option enabled, expect reduced performance.
GEOM_RAID: Intel-ee147409: Array Intel-ee147409 created.
GEOM_RAID: Intel-ee147409: Disk ada4 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-ee147409: Subdisk Volume0:0-ada4 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-ee147409: Disk ada5 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-ee147409: Subdisk Volume0:1-ada5 state changed from NONE to ACTIVE.
GEOM_RAID: Intel-ee147409: Array started.
GEOM_RAID: Intel-ee147409: Volume Volume0 state changed from STARTING to OPTIMAL.
GEOM_RAID: Intel-ee147409: Provider raid/r0 for volume Volume0 created.

Posted on

Jailing FreeBSD 4 on FreeBSD 10

We have an in-house application that was written for FreeBSD 4 and antediluvian versions of PHP, Perl, OpenSSL, and so forth. Most of the features have migrated into other applications, but a few critical functions remain.

An old operating system isn’t sufficiently bad, though. The hardware terrifies me. Not only is it over a decade old, it’s repurposed desktop hardware.

Virtualize it? Maybe. But device drivers have changed over the intervening decade, and a ten-year-old de(4) or fxp(4) driver works poorly on any of my virtualization systems. Virtio is right out.

Port it to a current OS, PHP, and perl? That would be a painful prospect if I knew what I was doing. I’m a sysadmin, not a programmer. I have no bloody clue what I’m doing.

But theoretically, FreeBSD 4 systems should run almost unchanged in a jail on FreeBSD 10. Can they? Let’s find out!

First I tarred up the entire 4.10 system, except for /proc. (Yes, FreeBSD 4.10 used /proc. Those were the days, eh?) I did no preparatory cleaning, and even included port work directories, /usr/obj, /var/tmp, and so on, as I have NO idea what I might need. Yes, I’m sure I can find a PHP 5.0.whatever tarball out on the Internet, but that would involve work.

Create my jail directory, and untar the copied server

# mkdir /var/jail/oldserver
# cd oldserver
# tar -xvpf $HOME/oldserver.tgz

Be sure to use the -p flag, to preserve permissions.

Now to edit some configuration files, to change this system from hardware to jail.

  • Check rc.conf. Remove functions handled by the host server: firewalls, timekeeping, SSH, and so on. This jailed host only needs the functions that directly support the application (the Apache web server and the associated MySQL database).
  • Check /etc/fstab for anything you need to retain, like NFS filesystems or /proc. Comment out everything else.
  • Remove the old /dev. You cannot use FreeBSD 4 device nodes on a FreeBSD 10 host. (I spent an hour repeatedly bashing my head on the brick wall intensively testing these two sentences, so you can rest assured that it’s true.)

    Now on to the jail server. FreeBSD 9 and above has a jail-specific configuration file, /etc/jail.conf. I define some basic characteristics here.

    While that’s running, do some basic jail setup. I would normally recommend ezjail, but this is a fairly special case: no ZFS, the server will never be upgraded, and I specifically want a very minimal system. Also, ezjail doesn’t seem to have been updated for the Jail New World Order. So I’ll configure this the hard way, which isn’t terribly hard.

    FreeBSD 9 and above has a jail-specific configuration file, /etc/jail.conf. Here’s a configuration for my old server.

    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";

    oldserver {
    path = /var/jail/oldserver;
    host.hostname = oldserver.mwlucas.org;
    ip4.addr = 10.0.16.31;
    interface = vtnet0;
    };

    Now for the real test:

    # service jail start oldserver
    Starting jails: oldserver
    #

    Is it that easy?

    Of course not. Several processes didn’t start. I had to edit some configuration files to account for the change of IP address. That’s pretty minor, though.

    To log into the jail, get the jail ID and run a shell prompt in that server.

    host# jls
    JID IP Address Hostname Path
    7 10.0.16.31 oldserver.mwlucas.org /var/toolkit
    host# jexec 7 /bin/tcsh
    oldserver#

    The advantage to hosting this ancient system as a jail is that I can use the host operating system to control access to the legacy application. I don’t have to use the ancient IPF included on FreeBSD 4.10; I protect this host with PF from FreeBSD 10. (Admittedly, there’s newer PF out there, but even the older version in FreeBSD 10 is better than IPF.) FreeBSD 4.10 includes OpenSSH 3.5, which has roughly the same security as Windows XP. To log into the old server now, you must authenticate to the up-to-date FreeBSD 10 system and get a jail command prompt.

    The disadvantage to this system is that you can’t run ps(1) inside the jail. The ps command reads kernel data structures. While the FreeBSD 10 kernel includes FreeBSD 4 compatibility, that compatibility doesn’t extend to ps(1) and similar commands. You must use ps(1) from the jail host.

    That’s an acceptable trade-off, if it means I don’t have to touch actual PHP code. That stuff has cooties.

    • Posted on

    Amazon gets it wrong

    Amazon sends us Kindle owners an email every few days saying “You bought this and that, so we think you’ll like these items.” They’re right an annoying large portion of the time, but when they get it wrong they really blow it.

    Like I'd buy any of these
    Amazon Recommendation Failure

    Okay, yes, the book of PF is pretty good. But I have a copy. And I actually paid cold hard cash for the FreeBSD Design & Implementation book, and waited in line to get it signed. Either of these would be excellent gifts for the BSD aficionado in your life, except that they probably already have them.

    But the others? The author has never said a nice word about me. In fact, every time I get near him he calls me a lazy good-for-nothing who ought to be working harder, better, and smarter, eating less, and in general doing more for humanity. Flavored with a whole lot of really bad language. One of these days, I’m going to punch that bastard in the face.

    The good news is, Amazon doesn’t know everything about us. Yet.

    Now, if you’ll excuse me, I’m off to buy everything I need for the next month via my Amazon Prime subscription.

    Happy holidays, everyone!

    Posted on

    Sudo Mastery and DNSSEC Mastery now at OpenBSD bookstore

    The official OpenBSD bookstore now carries both DNSSEC Mastery and Sudo Mastery print. If you buy the print from them, you get the ebook as well. These books sell for list price, but sales support the OpenBSD Project.

    I am not donating all my proceeds from these books to OpenBSD, however. I did that with SSH Mastery, because the OpenBSD guys kind of take it in the shorts with OpenSSH. One of my goals is to have the ability to make a living writing without needing survival tips like “There’s great cardboard boxes behind the appliance store” and “Get to Qdoba right after they close, because the night guy will give you a bag rather making you scrounge in the dumpster.” But the negotiations on this were pretty strange:

    Me: Sorry, guys, but I have to charge a couple bucks for myself on this.

    Them: No, you don’t. You charge us what you would charge any other direct order.

    Me: Oh, I can do better than that for you.

    Them: No, we want to pay what anyone else would pay.

    Me: Really, a discount isn’t a problem.

    Them: TAKE THE MONEY. DO NOT ANGER US.

    Me: [gulp] As you command.

    The end deal they got is better than what I charge Amazon. (Or, to be more accurate, it’s a better deal than the tithe Amazon extracts before condescending to carry my books.) If you want TWP books in bulk quantities, let me know.

    And if your company uses OpenSSH, get them to donate something. OpenSSH has ridiculous market share, and the developers basically work for tips. You wouldn’t leave a restaurant after a fabulous meal without tipping the wait staff, would you? Then why would you do that with software?

    Posted on

    On asking me to write for you

    [posted for later reference]

    In the first eleven days of December 2013, I have received eight requests for me to write for a periodical such as a web site or a magazine. This is nice. I struggled for many years to get published. To have publishers knock on my door and ask for my work gives me a certain warm fuzzy feeling. They’re trying to fill in their 2014 editorial calendars, and want me to be part of it? That’s kind of cool.

    There’s only one problem: they want to pay me with a subscription. The more generous ones offer advertising space. I address this in my FAQ, but it seems these people either don’t read the answer, don’t comprehend the answer, or think the answer doesn’t apply to them.

    Here’s an explanation with more detail.

    My writing time is completely occupied, either with work that I expect will return financial rewards or “writing of the heart” — projects that I really want to do, but that I accept will not pay.

    Generally speaking, if you’re contacting me with a request to write for you, you expect to make money off of my writing. That makes this a business transaction. This means I expect to get paid an amount that is roughly equivalent to the amount I would make if I expend that amount of effort on other paying channels. A thousand-word article is almost certainly more than $50 of my time.

    But it’s also important to not be a jerk. The world is a small place.

    From now on, I’ll answer these requests with a form letter.

    Hi,

    Thanks for your interest.

    At this time, I am completely occupied with paying writing work, so I cannot take your offer. But thanks for thinking of me.

    Regards,
    ==ml

    I’m not a total mercenary. I put a fair amount of technology content up in this blog, free for anyone who can use a search engine. But: I have a day job. My writing time is taken away from family and friends. I might choose to give up some of that time for someone. But that “someone” will be a person, not a business.

    I know other people will write for these periodicals. Someone always will. But that’s their choice. I choose otherwise.

    Posted on

    FreeBSD authentication against Samba 4 LDAP

    After years of only needing central auth for Unix-like systems, I need to integrate Windows clients into my auth mix. Rather than munging my current OpenLDAP directory to contain Windows information, I elected to migrate to Samba 4. Samba 4 can act as a Windows domain controller and also exposes an LDAP interface for Unix clients.

    I assume that you’ve read the FreeBSD and Samba documentation on LDAP auth. This article is meant as a bridge between the two information sets.

    Most of the tutorials out there cover using LDAP and Kerberos with Samba. Kerberos requires that all hosts be in a single domain. My employer manages hosts within a variety of domains and business units, so Kerberos is a no-go. I need to use pure LDAP authentication.

    I configured my samba4 domain and ensured that Windows clients could join the domain and that the general Microsoft-esque features worked, such as failover to a backup domain controller. Once it appeared that everything worked, I set up a couple of OpenLDAP proxies exactly by the Samba documentation. (My domain controllers are in private address space, and I’m not willing to expose them to the larger network.)

    Then I created an account for services to bind with to perform basic queries. My binding account is called unixstuff. It’s not a member of any AD groups.

    Then I can configure OpenLDAP on the client. Install the various LDAP utilities as in the FreeBSD documentation.

    Configure the OpenLDAP tools to query your directory first. I use a private CA, so I set assorted TLS options.

    BASE dc=internal,dc=mwlucas,dc=org
    URI ldap://snarky.mwlucas.org
    ssl start tls
    tls_cacert /usr/local/etc/ssl/mwlucas.crt

    At this point I should be able to use ldapsearch on the Samba directory.

    # ldapsearch -WxD "cn=unixstuff,cn=users,dc=internal,dc=mwlucas,dc=org"
    Enter LDAP Password:
    ...

    If you enter the correct password, the directory should spill its guts.

    Once that works, we can configure LDAP authentication in /usr/local/etc/ldap.conf.

    host snarky.mwlucas.org
    base dc=internal,dc=mwlucas,dc=org
    ldap_version 3
    binddn cn=unixstuff,cn=users,dc=internal,dc=mwlucas,dc=org
    bindpw WhyHardCodePasswords

    pam_filter objectclass=posixAccount
    pam_login_attribute uid
    pam_member_attribute member
    pam_lookup_policy yes
    nss_base_passwd cn=users,dc=internal,dc=mwlucas,dc=org
    nss_base_group cn=users,dc=internal,dc=mwlucas,dc=org
    pam_groupdn cn=sysadmins,cn=users,dc=internal,dc=mwlucas,dc=org

    # Services for UNIX 3.5 mappings
    nss_map_attribute homeDirectory unixHomeDirectory
    nss_map_objectclass posixAccount User
    nss_map_objectclass shadowAccount User
    nss_map_attribute uid msSFU30Name
    nss_map_attribute uniqueMember msSFU30PosixMember
    nss_map_attribute userPassword msSFU30Password
    nss_map_objectclass posixGroup Group
    pam_filter objectclass=User
    pam_password ad

    With this set, and LDAP activated in /etc/nsswitch.conf, I can now “getent passwd” and “getent group” and get responses from the directory.

    That still leaves PAM. I’m no PAM expert, but I’ve hacked together something that permits LDAP access, and falls back to the local password file when the LDAP servers are unavailable. Here’s my /etc/pam.d/system:


    auth sufficient /usr/local/lib/pam_ldap.so
    auth required pam_unix.so no_warn try_first_pass nullok

    account required pam_login_access.so
    account sufficient /usr/local/lib/pam_ldap.so
    account required pam_unix.so

    session required /usr/local/lib/pam_ldap.so
    session required pam_lastlog.so no_fail

    password required pam_unix.so no_warn try_first_pass

    This gets you basic access. Realistically, though, users want a home directory. Install pam_mkhomedir and enable it on a per-service basis.

    auth sufficient /usr/local/lib/pam_ldap.so no_warn
    auth required pam_unix.so no_warn try_first_pass

    account required pam_nologin.so
    account required pam_login_access.so
    account required pam_unix.so
    account required /usr/local/lib/pam_ldap.so no_warn ignore_authinfo_unavail ignore_unknown_user

    session required pam_permit.so
    session required /usr/local/lib/pam_mkhomedir.so

    password required pam_unix.so no_warn try_first_pass

    With this set, users in the “sysadmins” group in AD have shell access to the servers. So far, this works well. But if you’re a Samba, LDAP, or PAM expert and see a problem, please let me know in the comments.

    Posted on

    FreeBSD-update seems to hang on 10.0-BETA2

    I’m setting up a new FreeBSD web server. As 10.0 is just around the corner, I installed 10.0-BETA2. BETA4 is out, so it’s time to upgrade.

    # freebsd-update -r 10.0-BETA4 upgrade
    Looking up update.FreeBSD.org mirrors... 5 mirrors found.
    ...

    That all looks good. Then I installed the update

    # freebsd-update install
    ...

    And the install hung. Overnight. Why?

    Turns out that there are multiple errata on freebsd-update. While you can upgrade to BETA2-p2, you can’t jump directly to the next version.

    So you need to do:

    # freebsd-update fetch
    # freebsd-update install
    # reboot
    # freebsd-update -r 10.0-BETA4 upgrade
    # freebsd-update install

    When freebsd-update appears to hang, use ^T to print the current status.

    # freebsd-update install
    Installing updates...
    ^T
    load: 0.17 cmd: install 2796 [nanslp] 156.82r 0.01u 0.02s 0% 6220k

    Here we see that the install process, PID 2796, is running. Wait a few minutes and try it again.

    ^T
    load: 0.16 cmd: install 2803 [nanslp] 26.09r 0.01u 0.01s 0% 6160k

    There’s been no visible output, and install is still running, but it’s now PID 2803. It’s a new install process. The update is proceeding, if not as quickly as you’d like.

    So, be patient and let the machine do its work.

    Posted on

    Wanted: BSDCan Volunteers

    Gossip, rumor, innuendo, and Twitter say that the BSDCan 2014 Call for Papers should be out in the next few days. (I trust the first three sources more than the last, but what the heck, let’s go for completeness.)

    For the last ten years, Dan Langille has single-handedly coordinated BSDCan. In that time, BSDCan has become one of the critical events of the BSD community. It’s the largest, most consistently held gathering of BSD folks on this side of the planet.

    Last year, Dan asked for additional help organizing, coordinating, and running BSDCan.

    We had some great volunteers for the program committee, and several attendees offered their tender hides for assorted roles. A lot can change in a year, though, and not everyone in the world was in the room at the time, so I’m throwing this out to the general public.

    We could use some additional help with the grunt work. It would be nice to have someone assigned to sit in each presentation room to make sure that the presenters have everything that they need and that everything goes reasonably well. One of the big time sinks is coordinating arrangements with speakers. Broken up between a few people it’s not a huge task, but it requires a certain attention to detail.

    We’d also like to have someone local to the University of Ottawa to indoctrinate introduce more students into the BSD community.

    Want to help? Sign up for the BSDCan-volunteers mailing list.

    Posted on

    Moving mailboxes from Courier/Maildir to DirectAdmin/dovecot/Maildir

    I have an old mail server running Postfix and courier-imap. We want to split our customers off onto their old server, preferably something with a pretty pointy-clicky interface so that they can manage their own accounts. (Yes, people do still buy email service these days.)

    The old server runs FreeBSD, postfix, and courier-imap. The new server runs FreeBSD 9.2 with DirectAdmin. DirectAdmin is a canned package that puts a customer-friendly front end on a whole bunch of standard Unix software. (Why FreeBSD? ZFS. When my customers fill up the hard disk, I can just “zfs send” the whole machine to bigger disk.)

    The good news is, both my old server and the new one use Maildir. The bad news is, they’re arranged slightly differently and have somewhat different file formats. Tools exist to do 80% of the migration for you, but my setup has a few edges.

    I suspect that nobody else has my exact legacy setup, and my readers don’t care about this level of detail. I also suspect that the migration project will last weeks and I will completely forget how I did it in between individual customer migrations, so I’m writing this anyway. But maybe someone else can learn something from it, even if it’s just “man, I would never hire this Lucas character.”

    Create your email domain and all the user accounts in DirectAdmin. I don’t know any way to migrate encrypted virtual user passwords from Postfix into DirectAdmin, so I’ll need to create new passwords for the users. That’s not uncommon for a packaged mail server migration, so my users will live with it. This is a tedious point-and-click operation, but point-and-click is the point of DirectAdmin, so give the job to a minion and get it done.

    But how to transparently migrate the mailboxes from the Courier server to Dovecot?

    On the old server, the mailboxes are in /disk2/mail/vhosts/domainname/account/. On DirectAdmin, the mailboxes are in /home/userid/imap/domain/account/Maildir/. I can tar up the old accounts and drag the whole directory over to the new server, but I need to massage the directories so that they’re arranged properly.

    Courier and Dovecot both use files to store mailbox state information. IMAP state is important. Without it, users will re-download every single message. I don’t care about the bandwidth or the disk I/O this will cause, but the users will complain. Complaints lead to meetings, and meetings lead to the Dark Side. There’s a script to convert from one to the other. Dovecot also has good migration documentation, I recommend you read that before doing your own migration.

    The file ownership needs correcting. DirectAdmin creates user accounts for each customer (more or less), and that account needs to own the email files for the various domains.

    So, what am I willing to do manually?

    I will manually tar up the domain’s mail directory, copy it to the destination server, and untar it in /home/userid/imap/domain/. And I’m willing to provide the username that should own the files. I’m sure both of these could be automated, but I don’t have enough accounts to migrate to do this correctly.

    I’m not willing to move files around or run migration scripts. Because these are things that human beings mess up.

    There’s ways to do this in shell scripts, but I’m using Perl. Because while my Perl makes small children cry, my shell scripts feature in rituals praising Nyarlathotep.

    So, the script:

    #!/usr/bin/perl
    #migrate from old mail server to new
    #run from domain directory
    #Takes one argument, the username expected to own the files when done.

    unless ($ARGV[0]) {
    die "\nNeed a username to own files!\n\n";
    }

    opendir (USERS, ".") || die "Cannot open current directory";

    foreach $user (readdir (USERS)) {

    next if ($user =~ /^\.{1,2}$/);
    print "Converting $user...\n";
    mkdir ("$user/Maildir") || die "Cannot make Maildir";
    opendir (USERDIR, "$user") || die "Cannot open user directory";
    foreach $file (readdir (USERDIR)) {
    next if ($file =~ /^\.{1,2}$|Maildir/);
    print "Moving $user/$file\n";
    rename ("$user/$file","$user/Maildir/$file")
    || die "Cannot move $file";
    }
    #next user
    }
    print "Now performing courier-dovecot migration...\n";
    system ("courier-dovecot-migrate.pl --to-dovecot --recursive --convert");
    print "Fixing ownership...\n";
    system ("chown -R $ARGV[0]:mail .");

    Is this pretty trivial? Yep. But the most error-prone part of any process is the part I do. The more of the migration the machine does, the fewer screw-ups in the migration.

    Posted on

    mug.org OpenBSD talk on-line

    Last night’s talk on OpenBSD is now live in the mug.org channel.

    Part 1
    Part 2

    UPDATE: All in one.

    Among other things, I compare OpenBSD to Richard Stallman and physically assault an audience member. (Brian was a very good spots, and learned an important lesson about volunteering, e.g.: don’t.) We also talk long long time, memory randomization, PF, BSD license versus GPL, Microsoft, and other OpenBSD stuff.

    And, of course, the importance of the VAX.