Category: Tech

Many of my FreeBSD servers are not behind a firewall.  They sit naked on the Internet, and I protect their services with PF.  I have several “trusted” networks, and want to use them in macros.  Keeping track of serveral networks in a macro is error-prone, however.  Previously, I used macros like this one:

#lucas_house=10.20.20.0/28
#main_office=192.168.1.0/25
#monitor=17.16.1.1
#boss_house=10.20.30.0/24
mgmt_networks ="{ 10.20.20.0/28, 192.168.1.0/25, 172.16.1.1, 10.20.30.0/24 "}"


This meant entering each IP address twice.  Complicated numbers hurt my feeble brain, and the result is errors.  Entering each address multiple times is begging for an error.  I found that you can nest macros, however, with careful placement of single and double quotes.

lucas_house='"10.20.20.0/28"'
main_office='"192.168.1.0/25"'
monitor='"17.16.1.1"'
boss_house='"10.20.30.0/24"'
mgmt_networks ="{" $lucas_house $main_office $monitor $boss_house "}"

Note that each address is in single quotes (‘), enclosed by double quotes (“).  In the mgmt_networks macro, put double quotes around the enclosing brackets. This is in the man page example, but you have to look very closely at it.

I can then allow SSH, SNMP, SIP, etc, from my management networks to the server, and my addresses will be consistent.

I’ve previously written about using HAST to replicate a ZFS between servers, using CARP devd events to trigger userland changes, and combining them to automate ZFS failover.  The scripts reference a master-process.sh and slave-process.sh used to manage processes, and people have asked to see them. Continue reading “process management with CARP/HAST failover”

I need a firewall cluster for $DAYJOB.  I could grab any BSD, throw it on commodity hardware, and make it work.  I’ve done that.  Repeatedly.  I could plug OpenVPN into Radius.  I could set up CARP and rsync-based config replication.  But I decided to try pfSense instead.  Here’s why, and what happened… Continue reading “pfSense experiences”

OpenNebula users know that NFS is just too slow for virtual machine disk images.  Fiber Channel works, but is too expensive for me.  Rather than deal with disk image speed issues, I’m using NFS on ZFS for file storage and booting my systems diskless.  Diskless servers have a lot of advantages, but speed isn’t one of them.  This is fine for most applications, but a few things (databases come to mind) perform better on a speedy disk.  I want the ability to use diskless machines where appropriate, but use cheap networked disk when necessary.  Ideally, I want iSCSI on top of ZFS.  Short of ideal, I’ll take iSCSI any way I can get it.  I want the virtualization server to attach to the iSCSI target, and then offer that target to the VM as if it was a local disk.

There’s an alpha one-iSCSI-target-per-VM transfer manager driver.  It’s intended for a Linux iSCSI server, which I don’t have and don’t intend to run.  Instead, I have a stack of cheap NAS appliances.  Here’s how I got one target per VM running in my OpenNebula instance. Continue reading “opennebula with one iscsi target per VM”

I use KVM and OpenNebula on Ubuntu for virtualization. Getting such a cluster up and running is easy, but making it perform well takes much more work.  Many times, the statement “my virtualization cluster works well” is equivalent to “I’m not paying attention.”  My FreeBSD hosts help point out problems, though.  All of my FreeBSD servers send me a daily email to tell me they’re still alive and to point out potential issues.  That’s how I found out I was getting network collisions on my virtualized hosts, and here’s how I investigated them. Continue reading “Network collisions running hosts under KVM”

Post summary:  Get wireshark.  Use it.  It might not solve your problem, but it will tell you who to blame. Continue reading “debugging iSCSI”

I’m leaving my getting hit in the head lesson when the boss calls.  Some unmentionable orifice is firing DOS attacks at a couple of our SIP servers.  My mission, should I choose to accept it, is to find and block the attackers.  (Should I choose to not accept it, then my mission will be to listen to Fearless Leader whine about it.  I can’t stand whining.)  Fortunately, I have flow data for one of the servers under attack. Continue reading “Finding a SIP DoS attack via flow analysis”

Earlier I discussed using HAST to mirror a ZFS across two machines.  I also discussed using devd with CARP interfaces, to run a script when a machine changes between master and backup.  Now let’s glue these together to synchronize a ZFS switch with CARP state, and to run scripts when the system becomes the master or the backup. Continue reading “Automated CARP/HAST Failover”

In my last post I discussed using HAST with ZFS.  That tells you how to replicate a filesystem back and forth between two machines.  That’s nice, as far as it goes, but I want automatic failover.  Clustering.  I want to wake up in the morning to a message that says “machine 1 failed, machine 2 took over, and nobody noticed” instead of a lot of messages from angry customers.  The standard FreeBSD failover mechanism is CARP, the Common Access Redundancy Protocol.  Here’s the basics of CARP. Continue reading “CARP and devd on FreeBSD”

There’s a nice tutorial on using HAST (Highly Available STorage) with UFS and ucarp.  That’s very nice, but in my failover scenario I can’t use UFS; a fsck would take too long, and a background fsck would be most likely to lose the data I’m most likely to need.  And FreeBSD comes with a kernel-side CARP implementation; why would I use the userland implementation instead?  So: the tutorial is great, except it doesn’t do what I want.  I’ll attack this problem in two phases:  one, get HAST with ZFS running, and experiment with it.  Two, get CARP failover to trigger HAST failover automatically.  (I believe I can use devd for CARP-initiated failover, but I’ll need to do further research on that.  That’ll be another posting.)  Today I’m experimenting with HAST and ZFS.  Continue reading “HAST and ZFS”