Category: bsd

Posted on

How to Lose your Job with SSH, part 2

Like last week’s How to Lose your Job with SSH, Part 1, a less dramatic title for this would be Dynamic Port Forwarding with SSH, but that’s dreadfully dull.

Many corporations try to tightly secure their network. Connections to the outside world are strictly limited. If you have a single open TCP/IP port to the outside world, however, you can leverage this into blanket Internet access by using your SSH client as a SOCKS proxy. This is called dynamic forwarding.

SOCKS is a generic TCP/IP proxy. SOCKS tunnels arbitrary protocols, as long as the client program supports SOCKS. Many clients, including the major Web browsers, do.

To do this, you would SSH from your client on the protected network to a server on the public Internet. The client opens a SOCKS proxy on a TCP port on the local machine. When a client connects to this proxy, the traffic is forwarded across the Internet to the SSH server. The SSH server processes the request and feeds the data back to you. It’s not a terribly fast proxy, but it is encrypted and it will bypass the corporate firewall and Web proxies.

Use the -D flag to tell the OpenSSH client to use dynamic port forwarding. (You can do the same thing with PuTTY, but that’s a separate walk-through.)

$ ssh -D localaddress:localport hostname

If you don’t specify a local IP, the client automatically binds to 127.0.0.1.

Here, I open a dynamic proxy on port 9999 to the server pride.blackhelicopters.org

$ ssh -D 9999 pride.blackhelicopters.org

Leaving the SSH session up, now go to the Web browser on the local machine. Somewhere in the browser preferences you’ll find a place for proxy servers. Tell your browser it has a proxy at 127.0.0.1 on port 9999. Save the settings and start browsing the Internet. Your Web browsing will be slow, but it’ll get you on the Internet.

If I list a network-facing IP on my client, anyone who can connect to that port on the client can use my proxy.

$ ssh -D 10.10.10.105:9999 pride.blackhelicopters.org

This is, of course, a violation of the security policy at these security-sensitive companies. Doing it will get you fired.

As a network administrator responsible for such an environment, what can you do about this?

  • If you run an external SSH server that your users need to log into, disable SSH forwarding with the AllowTcpForwarding option in sshd_config.
  • Watch your traffic. See how much traffic various protocols use on your network. This will not only help you catch SSH tunnels, it will help you catch any other sort of tunnel. You must know what is normal before you can catch strangeness, however.
  • You could forbid external SSH, and only permit protocols that can be proxies, such as Telnet.
  • You can set up a legitimate SOCKS server, with logging, and require all traffic pass through it. Anyone tunneling large amounts of traffic through the server will stand out in the logs.

    • Most of these solutions require the network administrator to know what happens on his network. You must have basic network awareness, in advance, or you’ll never find these people.

    And if you’re an end-user who creates his own SOCKS proxy in such an environment, gets caught, and gets fired for it, do let me know. I can always use a good laugh at someone else’s expense.

    Posted on

    How to Lose your Job with SSH, part 1

    A less sensational title for this post would have been “SSH Remote Forwarding,” but that’s not nearly as fun.

    I used to be responsible for one of the few entry points into a global network. The company had actual manufacturing secrets — their products included various machines of war. We had internal firewalls to protect sites from each other, even when the site didn’t have Internet access. All Internet connections had to go through proxies. We did not allow external DNS to reach the desktop. If you typed ping google.com on your desktop, you’d get a “host unknown” error. The company had invested in VPN technology that blocked all but approved clients, and only permitted clients on the approved list if they were running the approved anti-virus scanners and other security software.

    I was frequently asked to open direct Internal access for random applications. Most of these requests were rejected unless the user could explain what they wanted and why it was business-critical. Some of the people who asked were technically literate, and became indignant when I rejected their request for outbound SSH to external servers or equipment. After all, SSH is “Secure Shell.” It says secure RIGHT IN THE NAME. They just want to check their personal email. How could I possibly reject this eminently sensible request?

    I’ve since had left this job, but I’ve had the same discussion more than once afterwards.

    Most SSH users have no idea of SSH’s flexibility. Arbitrary SSH connections are a nightmare for maintaining any sort of secure information perimeter. Remote port forwarding is one reason why.

    When most people mention SSH port forwarding, they’re thinking of local port forwarding. You forward a TCP port on your client to a TCP port on the SSH server. This lets you, say, tunnel SMTP inside SSH, so your client can relay mail through your server without any complicated Sendmail rules.

    Remote forwarding lets you do the reverse: forward a TCP port on the SSH server to a TCP port on the SSH client.

    Suppose I permit a user to SSH to an external server. The desktop is behind a NAT. There are no port mappings from the NAT to the desktop; it has the same connectivity you might give a secretary, except for the outbound SSH access to a single host. That user sets up remote port forwarding from a TCP port attached to the server’s public IP to the client’s SSH daemon. I’m using an OpenBSD desktop, but you can get SSH clients for most other operating systems, including Windows. I’m using OpenSSH, of course, but most clients (including PuTTY) does remote port forwarding.

    To do remote port forwarding in OpenSSH, run:

    $ ssh -R remoteIP:remoteport:localIP:localport hostname

    If you don’t specify an IP address to attach to the SSH server, the server attaches to 127.0.0.1. (You can also skip the first colon in this case.)

    My desktop runs sshd. I want to attach port 2222 on the SSH server pride.blackhelicopters.org to port 22 on my SSH client using remote port forwarding.

    client$ ssh -R 2222:localhost:22 pride

    I leave this connection up and go home. Perhaps I run top in the command window, to prevent the SSH session from timing out at the corporate firewall. Now at home, I log into the SSH server and run:

    pride$ ssh -p 2222 localhost

    My SSH request to the local machine will get tunneled inside the existing SSH connection out of the network. I will get a logon prompt for my client inside the secure network. When I can access one supposedly secure machine, I can start ripping data out of the local file servers and maybe even access other internal sites. All of the trouble the company expended to prevent unauthorized access is now moot.

    If you attach the remote port forwarding to the server’s public IP, then anyone on the Internet can attack your desktop’s SSH daemon. People are attacking SSH servers, even on odd ports.

    Of course, remote port forwarding in this environment would violate company policy. If caught, you’d lose your job. But to catch this abuse, the network administrator would need to realize that large data transfers were taking place in off hours over these limited-use channels. You could, say, use flow analysis to write automated reports that notice and alarm when large amounts of traffic pass over these “rarely-used” channels. You’d have to be a real bastard to think of that. But the reports are easy to write, and the look on the abuser’s face when you confront them with graphs and numbers is priceless.

    The point is, the next time your employer’s network administrator rejects your sensible request for SSH access to your home server, don’t be too hard on the poor slob.

    Want more SSH fun? Check out the book recommended by the OpenSSH project, SSH Mastery.

    Posted on

    creating FreeNAS 8 iSCSI target

    I didn’t find an up-to-date tutorial on how to set up a FreeNAS 8 iSCSI target, so I took notes as I set one up. Figuring this out by brute force is fairly straightforward: just keep adding iSCSI stuff until you can actually discover iSCSI targets. But for those of you who value your time, here’s how you do it.

    1) Go to services, enable iscsi. Under iSCSI, choose Target Global Configuration. In Discovery Auth Method choose CHAP. Leave all other settings unchanged

    2) It seems that FreeNAS doesn’t yet let you export a whole drive via iSCSI; you must create an extent. Under iSCSI, select Extents, then Add Extent. Name your extent, give it a path, and set a size.

    3) On the Target Global Configuration, on the “authentication” tab, create an iSCSI user and assign it a secret

    4) You need an iSCSI Portal. Click the Portals tab. This is where you define the iSCSI target. The default is to listen for iSCSI connections on all IP addresses, on port 3260. If you have multiple IP addresses, and want separate portals for each, be more specific.

    5) You must authorize initiators. Choose “Authorized Initiator,” then “Add Authorized Initiator.” List the IP subnet of your initiators under Authorized Network.

    6) Go to the Targets tab. Add a target.

  • Give the target a name and and an alias.
  • The type will be “disk.”
  • flags read-write
  • portal group ID is the number of the portal (probably 1)
  • initiator group ID is the number of the authorized initiators you created (probably 1)
  • Set auth method to CHAP
  • authentication group number from the user you created (also probably 1)

    • 7) Under “Associated Targets,” attach the extent to the target.

    Now go to your iSCSI initiator and log in.

    To create new iSCSI targets, create a device extent, add a target for that extent, and attach the extent to the target.

    Posted on

    add PF to FreeNAS 8

    I won’t put any system on the Internet without some sort of packet filter or firewall. Especially not a file server containing important data. While I have a FreeNAS 8 box, it doesn’t come with a firewall. FreeBSD includes three firewalls, IPFW, IPF, and PF. Any one of these would do, but I’m most comfortable with PF, so I decided to add PF to my FreeNAS install.

    This will not add a firewall to the GUI. You must manage your rules remotely. The FreeNAS backup will not back up your rules. The good news is, once your firewall works the rules don’t change very frequently.

    First, get the appropriate kernel module. You’ll need the ISO for the version of FreeBSD that your FreeNAS is based on. FreeNAS 8.0 is based on FreeBSD 8.2. I’m running the amd64 version of FreeNAS, so I need the amd64 version of FreeBSD 8.2. The version must match as exactly as possible: your 8-stable amd64 desktop might work, but it might make your FreeNAS box crash and die.

    Mount the ISO.

    $ mdconfig -a -t vnode -f $HOME/FreeBSD-8.2-RELEASE-amd64-livefs.iso
    md0
    $ mount -t cd9660 /dev/md0 /mnt

    I previously installed FreeNAS on a 2GB USB stick. For this process, I shut down my FreeNAS box and mounted the USB drive on my FreeBSD machine. You could work directly on the FreeNAS image and re-install it on the USB drive, or even work directly on the live FreeNAS box. I’m assuming you’re working on a USB stick.

    $ mount /dev/da0s1 /mnt

    Start by getting the PF kernel module from the ISO onto your USB drive.

    $ cp /media/boot/kernel/pf.ko /mnt/boot/kernel/

    Now for the tricky bit. FreeNAS is based on the FreeBSD diskless system. (Obligatory plug: I did half a chapter on diskless operation in Absolute FreeBSD.) It uses the /conf directory to create a series of memory-based filesystem overlays, creating a MFS /tmp, /etc, and so on. This means that any changes you make to, say, /etc/pf.conf will not survive a reboot. You must edit the original versions of these files, in /conf/base/etc.

    In /conf/base/etc/rc.conf, add

    pf_enable=YES

    Put your pf.conf in the same directory. Configuring PF is left as an exercise for the reader.

    Any firewall changes made to a running system must appear in /etc/pf.conf and in /conf/base/etc/pf.conf. I suggest changing the base file, and copying changes to /etc.

    Posted on

    FreeNAS 8 on Intel SS4200-E

    The boss snagged an Intel SS4200 NAS as part of the corporate “redundant array of inexpensive crap” strategy. Basically, it’s a small, headless amd64 machine with 512MB RAM, four SATA ports, plus eSATA and USB. Add four 1TB hard drives, and you have a decent small storage device. I decided to try FreeNAS 8, so I could get ZFS, NFS, and iSCSI.

    FreeNAS 8 installs on a USB drive. I got a 2GB USB from Micro Center checkout counter, and escaped the store without purchasing anything else.

    Start by getting the FreeNAS full install, aka FreeNAS-8.0-RELEASE-i386.Full_Install.xz. Verify the checksum, then decompress the image.

    $ unxz FreeNAS-8.0-RELEASE-i386.Full_Install.xz

    Use file(1) to check it.

    $ file FreeNAS-8.0-RELEASE-i386.Full_Install
    FreeNAS-8.0-RELEASE-i386.Full_Install: x86 boot sector; partition 1: ID=0xa5, active, starthead 1, startsector 63, 953505 sectors; partition 2: ID=0xa5, starthead 1, startsector 953631, 953505 sectors; partition 3: ID=0xa5, starthead 0, startsector 1907136, 3024 sectors; partition 4: ID=0xa5, starthead 0, startsector 1910160, 41328 sectors, code offset 0x31

    My USB drive is available as /dev/da0. Copy the image to the drive.

    $ dd if=FreeNAS-8.0-RELEASE-i386.Full_Install of=/dev/da0 bs=5k

    While that’s running, console into the SS4200. The serial console connector seems to be a standard DB9 connector, so you can just pull one off of an old motherboard. If your null-modem cable doesn’t seem to work, try reversing the connector on the motherboard. I connected with 115200 8n1, no flow control.

    Go into the BIOS. Select Advanced, then go down to USB. Set your USB emulation type to “hard disk.” Then go into the boot order and have it boot from hard drives. Go to the Hard Disk Drives menu beneath that and make your USB device first on the list.

    If you don’t have the console connection, boot the SS4200 from USB by holding down the reset button (tiny black dot by the rear USB port) for about 2 seconds while powering on. The light will turn amber to show that it’s booting from USB. This will work for home use, although you’ll need to do this every time you reboot the NAS. You’re really better off setting the BIOS properly.

    Once dd finishes, plug the USB drive into the SS4200 and power up. The default FreeNAS IP is 192.168.1.250, but if you have a DHCP server it will grab a proper local IP. Check your DHCP server logs for a host called “freeNAS,” and point your browser at that address.

    Once I was in, I easily added data storage, creating two 931-GB ZFS mirrors.

    I successfully created group for my new account. I couldn’t create a user, but it’s a known problem with workarounds.

    Lastly I enabled the serial console by going to System->Settings. Oddly, there’s no setting to set the console speed. You can change the SS4200 serial port to 9600 in the BIOS, but that seems less than optimal. It’s a petty issue, however, as I can reconfigure my serial connection trivially.

    I found a FreeNAS 7/SS4200 tutorial buried inside the FreeNAS forums. While all the FreeNAS stuff has changed, it was quite helpful for setting up the SS4200 hardware and BIOS.

    Overall, FreeNAS 8 looks pretty. The real test will start when I put a bunch of diskless NFS and iSCSI clients on it and watch for stuff to break. It must have better iSCSI than the Iomega NAS’s I’ve been arguing with…

    Posted on

    desktop OpenBSD pf.conf

    I have an OpenBSD 4.9/i386 desktop sitting naked on the Internet, and found people poking at my TCP ports. While PF is enabled by default, it’s configured to permit everything except remote X11. I need a policy that will block incoming traffic from everywhere except a few key IP addresses, while allowing me to make any outbound connections I desire.

    mgmt="{192.0.2.0/24, 192.168.8.0/24}"
    set skip on lo
    block
    pass proto icmp
    pass from $mgmt to self
    pass from self to any
    block in on ! lo0 proto tcp to port 6000:6010

    Disabling ICMP globally is a bad idea. Lots of stuff will break. I could specify permitted ICMP types, but that would be a lot of work and probably break my connectivity to random in a really obscure but educational manner.

    I retained the X11 block, even for those known-good addresses, because if I’m trying to open an X11 connection to my home desktop without forwarding it over SSH, I am clearly solving the wrong problem.

    If I permit password-based SSH connections from “known good” hosts, such as my house and the office, then if one of those machines is penetrated, the intruder will be able to get into my machine. I protect my desktop by requiring SSH public key auth, even from trusted IP addresses. An intruder could be savvy enough to hijack my agent forwarding, but in that case it’s someone specifically targeting me.

    The current PF syntax is as close to painless as a firewall can be.

    Posted on

    Summer 2011 nonfiction project: OpenSSH

    I have a problem with Absolute OpenBSD, 2nd edition. It’s too big. The outline is 26 chapters. This brings the book close to 300,000 words, well over a thousand pages. I don’t want to write books that I don’t like. I don’t like huge books that I cannot comfortably read in the bathtub.

    One component of OpenBSD is OpenSSH. People have written books about OpenSSH, but they contain more information than most people need. (Not all, but most.) I write for the most common user, which means that those books are perhaps 2x-3x the size of what I would write. And OpenSSH is widely used outside OpenBSD: I could argue that it’s their most widely deployed component. Lots of people who will never use OpenBSD need a swift sharp smack with the OpenSSH clue-by-four.

    AO2e book needs some OpenSSH stuff in it, of course. But I have about 30,000 words of OpenSSH. For comparison, the smallest nonfiction book I’ve written is about 60,000 words. It’s too small for a traditional publisher.

    So I’m removing the intermediate-level OpenSSH material from AO2e, and doing a small independent book tentatively titled “OpenSSH: Your Next Steps.” It will cover the OpenSSH server, and the OpenSSH and PuTTY clients.

    Many people believe they know how to use SSH: they download PuTTY, enter a hostname, username, and password, and “poof!” — they’re secure. It’s not that simple. If you search for documentation and articles about using OpenSSH, you’ll find years and years of accumulated cruft. Many of the most visible articles and posts are obsolete. A single source of up-to-date information at an inexpensive price would find readers, and would let me satisfy my self-publishing curiosity. I’ve contemplated independent publishing for some time now, and this looks like a realistic opportunity.

    I’ll have to pay for copyediting, design, and so on. And I’ll have to manage the whole project. Worst case, I lose a lot of time and money, but develop a new appreciation for my publisher. (I still cannot see leaving them for major books like AO2e, however; it’s just too much work.)

    I believe that there’s a market for inexpensive, small, single-topic books, as Cisco Routers for the Desperate is one of my best-selling books on Kindle. Not bad for a two-year-old book. I could also make the case that if you want a book called “Cisco Routers for the Desperate,” you want it NOW. (The publisher could probably double the price without impacting Kindle sales, but that would be gauche.)

    I will do a community-based tech review. I’ll announce here when I’m looking for reviewers.

    Finally: at Austin Hook’s request, I’ll also do a print-on-demand version. (I’m not terribly interested in distribution of physical copies of such a short book, but the expense and annoyance of setting up POD isn’t that great, so why not?) Austin handles the sale and distribution of the various OpenBSD merchandise, so I have confidence he can manage this.

    As an independent, I cannot do pre-orders for books that don’t yet exist. There are Web sites that manage this for authors, but I really don’t want to bother. But Austin does that all the time, and he’s a trusted community figure. He will be the official source for pre-orders. While POD books need to be paid for in advance, the money doesn’t come to me until the books are ready to print and ship. This will get some extra cash in the hands of the folks who develop OpenSSH, which is a good thing.

    No, I don’t have numbers or prices yet. I’m taking a course on this stuff, as well as POD, though, so I’m not even trying to figure that out yet. I can assure you that the paper version will be more expensive than the electronic version, however.

    Will this succeed? There’s only one way to find out.

    UPDATE: I’m not shelving AO2e for the summer. I normally work on multiple projects simultaneously. When I’m sick of one, I work on the other.

    Posted on

    my .cwmrc

    I need a window manager that doesn’t take up desktop space with lots of icons and permits me to work without removing my hands from the keyboard. (I do use the mouse, mind you, but I don’t want to be required to use the mouse for routine tasks.) I’ve used cwm on OpenBSD as my window manager for about a year, and it’s fulfilled my needs perfectly well.

    I have made some changes. cwm is very easy to modify via the .cwmrc file. Here’s mine, with comments interspersed to show why I’ve made these changes.

    I want a narrow blue border around my windows, and I want the active window to be highlighted. Maximized windows leave a 180-pixel gap at the right-hand side of the screen, so my xclock remains visible. (Without a clock, I sink into an authorial coma and lose awareness of time, sound, light, and so on.)

    borderwidth 2
    color activeborder blue
    color inactiveborder darkblue
    gap 0 0 0 180

    My most heavily-used big applications are OpenOffice and Firefox. Both of these highlight text a word at a time when you use CTRL and an arrow key. By default, cwm maps CTRL-arrow to moving the pointer. I’m not going to train my fingers to make my applications work differently on OpenBSD than on any other operating system, so I needed to make cwm pass CTRL-arrow through to applications and use another key sequence for moving the pointer.

    bind CS-Left unmap
    bind CS-Right unmap
    bind CS-Up unmap
    bind CS-Down unmap
    bind C-Left unmap
    bind C-Right unmap
    bind C-Up unmap
    bind C-Down unmap
    bind 4S-Left bigptrmoveleft
    bind 4S-Right bigptrmoveright
    bind 4S-Up bigptrmoveup
    bind 4S-Down bigptrmovedown
    bind 4-Left ptrmoveleft
    bind 4-Right ptrmoveright
    bind 4-Up ptrmoveup
    bind 4-Down ptrmovedown

    The 4th option key (which probably appears as the Windows key on your keyboard) and an arrow now controls pointer movement.

    I experimented at length with my cwm configuration, and wanted to be able to make cwm reread .cwmrc with a keyboard shortcut, such as CTRL-ALT-r.

    bind CM-r reload

    I have a preferred screensaver, activated on a CTRL-ALT-DELETE.

    command lock '/usr/X11R6/bin/xlock -mode flow'

    Finally, I want to be able to start commonly-used programs via a right mouse click on the background. I don’t have to start them this way, but the option is convenient.

    command firefox /usr/local/bin/firefox
    command xpdf /usr/local/bin/xpdf
    command OOo /usr/local/bin/soffice

    With these modifications, cwm stays out of my way and lets me work.

    And while I’m babbling about cwm: when you spawn an SSH session with CTRL-. you can put SSH command-line arguments before the hostname. Very useful for when you want to enable, say, X11 forwarding for a particular session.

    Posted on

    OpenBSD, Firefox, and Flash

    An OpenBSD advocacy article led me to a mailing list posting on how to get Flash playing transparently in Firefox on OpenBSD. You could get Flash (and other clunky media formats) to work on Unix-like platforms that Adobe doesn’t support for some time now, using a combination of players in packages and nearly random hacks in Firefox. This process makes everything simple, however.

    I’m using 4.9/i386 and mozilla-firefox-3.6.13p3.

    Set a package path in your shell. A few tests with ping and traceroute showed ftp3.usa.openbsd.org is my closest mirror. I use tcsh, so my .cshrc has:

    PKG_PATH=ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.9/packages/i386/

    Then run:

    $ sudo pkg_add -r gecko-mediaplayer

    This installs a whole bunch of packages for playing not just Flash, but other complicated media formats.

    Browse to https://addons.mozilla.org/en-US/firefox/addon/flashvideoreplacer/ and install the plugin. The plugin handles replacing Flash with appropriate third-party players from packages.

    Restart your browser. And everything just works.

    Posted on

    FreeBSD iSCSI panic

    I woke up today to find a console with:

    panic: _mtx_lock_sleep: recursed on non-recursive mutex iscsi-io @ /usr/src/sys/modules/iscsi/initiator/../../../dev/iscsi/initiator/isc_sm.c:324

    The initiator is a FreeBSD-current amd64 from 8 May 2011. The iSCSI target is an inexpensive iomega NAS. Other hosts attached to this iSCSI NAS have also had errors, though. The errors clear when I reboot the NAS.

    Unfortunately, the FreeBSD box is a diskless system. Dumps aren’t exactly simple. While I heard some rumours about a network dump facility coming soon at the FreeBSD BSDCan devsummit, that’s the future.

    How to fix this?

    I attended the High Performance FreeBSD Clusters talk at BSDCan 2011. The presenter had originally used FreeBSD servers, then tried OpenSolaris to get better performance. He had OpenSolaris problems, but found that they could not access the bug information without a support contract. They’re now moving towards FreeBSD with EIT, and are happier.

    I intend to learn from their mistakes, and replace the iomega with a FreeBSD EIT server. I’ll keep the iomega for, say, a central ports and packages NFS server, where a reboot won’t impact my uptime.

    Why bother to blog this? So that the next poor bugger who gets this panic message gets at least one search engine hit.