Crime writer. Many of those crimes involve computers.
On Tuesday 27 May 2014, at 1pm EDT, I’ll be doing a live webcast on the O’Reilly community site.
The topic is “Beyond Security: Getting to Know OpenBSD’s Real Purpose.”
Anyone who has seen my mug.org talk already has an idea what angle I’ll take. Some of the slides will differ, and I’ll have some LibreSSL stuff in there as well.
I’ll take questions from the audience afterwards. You can be in that audience.
Thanks to various airline problems, we had an open spot on the BSDCan schedule. Bob Beck filled in at the last moment with a talk on the first thirty days of LibreSSL. Here are some rough notes on Bob’s talk (slides now available).
LibreSSL forked from OpenSSL 1.0.1g.
Why did “we” let OpenSSL happen? Nobody looked. Or nobody admitted that they looked. We all did it. The code was too horrible to look at. This isn’t just an OpenSSL thing, or just an open source thing. It’s not unique in software development, it’s just the high profile one of the moment.
Heartbleed was not the final straw that caused the LibreSSL fork. The OpenSSL malloc replacement layer was the final straw. Default OpenSSL never frees memory, so tools can’t spot bugs. It uses LIFO recycling, so you can use after free. The debugging malloc sends all memory information to a log. Lots more in Bob’s slides, but this all combined into an exploit mitigation technique countermeasure. Valgrind, Coverity, and OpenBSD’s randomized memory tools don’t catch this.
Someone discovered all this this four years ago and opened an OpenSSL bug. It’s still sitting there.
LibreSSL started by ripping out features. VMS support, 16-bit Windows support, all gone.
LibreSSL goals:
As an example, how does OpenSSH (not LibreSSL, but another OpenBSD product) do portable?
How does OpenSSL do portable?
The result? “Chthulhu sits in his house in #define OPENSSL_VMS and dreams”
Removed scads of debugging malloc and other nasties.
What upstream packages call and use them? No way to tell. LibreSSL makes some of the very dangerous options no-ops. Turn on memory debugging? Replace malloc wrappers at runtime? These do nothing. The library internally does not use them.
Some necessary changes that were implemented in massive sweeps:
OpenSSL used EGD for entropy, and faked random data. OpenSSL gathered entropy from the following sources:
In LibreSSL, entropy is the responsibility of the OS. If your OS cannot provide you with entropy, LibreSSL will not fake it.
LibreSSL is being reformatted into KNF – the OpenBSD code style. OpenSSL uses whatever style seemed right at the moment. The reformatting makes other problems visible, which is the point. More readable code hopefully means more developer involvement.
The OpenSSL bug tracking RT has been and continues to be a valuable resource.
OpenSSL exposes just about everything via public header files. Lots of the API should probably not be used outside of the library, but who knows who calls what? OpenBSD is finding out through constant integration testing with their ports tree.
The LibreSSL team wants to put the API on a diet so that they can remove potentially dangerous stuff. Their guys are being careful in this by testing against the OpenBSD ports tree. Yes, this conflicts with the “drop-in replacement” goal.
Internally, LibreSSL uses only regular intrinsic functions provided by libc. OpenSSL’s custom APIs remain for now only to maintain compatibility with external apps.
Surprises LibreSSL guys in OpenSSL:
So far:
LibreSSL has added the following cipher suites under acceptable licenses – Brainpool, ChaCha, poly1305, ANSSI FRP256v1, and several new ciphers based on the above.
FIPS mode is gone. It is very intrusive. In other places governments mandate use of certain ciphers (Cameilla, GOST, etc). As long as they’re not on by default, and are provided as clean implementations under an acceptable license they will include them. They believe it’s better people who must use these use them in a sane library with a sane API than rolling their own.
If you want to use the forthcoming portable LibreSSL, you need:
You can’t replace explicit_bzero with bzero, or arc4random with random. LibreSSL wants a portability team that understands how to make it work correctly.
LibreSSL’s eventual goals:
There’s lots of challenges to this. The biggest is stable funding.
The OpenBSD Foundation wants to fund several developers to rewrite key pieces of code. They want to sponsor efforts of the portability team, and the ports people track the impact of proposed API changes.
They will not do this at the expense of OpenSSH or OpenBSD.
The OpenBSD Foundation has asked the Linux Foundation for support, but the Linux Foundation has not yet committed to supporting the effort. (I previously said that they hadn’t responded to the request, which is different. The LF has received Bob’s email and discussions are ongoing.)
In Summary:
If you’re interested in supporting the effort, contact the OpenBSD Foundation. The Foundation is run by Bob Beck and Ken Westerback, and they manage all funding. (While Theo de Raadt leads the OpenBSD Project, he actually has nothing to do with allocating funding.)
Karl Lehenbauer, CTO of FlightAware, is giving an excellent BSDCan keynote: a retrospective of his BSD experience. As part of the mass of flight troubles plaguing North America this week, his flight to Ottawa was cancelled. He landed in Toronto at midnight last night.
I wouldn’t have blamed him for canceling the keynote.
Instead, he rented a car and drove to Ottawa. Overnight. After a bad day of travel. That’s about a four-hour drive.
Lehenbauer is clearly a man who keeps his promises.
Plus, he speaks very well despite not sleeping. Or maybe because he didn’t sleep. Whatever, he’s good.
The ports and packages summit was a lot more discussion of options as opposed to the state of items and future plans. A very dynamic session, where each of the dozen or so scheduled speakers was more “moderator of the moment.” Plus, I staggered in half an hour late, because breakfast was really really good.
But, in general, what happened:
I walked in on an overview of Debian packages. It’s always good to assess others’ work.
Discussions on dependencies.
Ed Maste on possibly using certificate transparency via X.509v3 extension, rather than creating our own signing infrastructure.
Using qemu BSD user mode for cross-building packages. Qemu still needs some work, and you can pitch in.
bapt: want to control what scripts can do, so arbitrary scripts can’t harm system. Have the system provide a utility that will let programs check config files or update a database, rather than run arbitrary scripts. Would also help with cross-building packages.
Cross-building is improving. Now nightly ARM crossbuild packages in test. Hopefully ready by EuroBSDCon.
PortCI: project for build cluster automation. Various port building processes are manual, such as testing and QAT. PortCI lets you manage these queues easily via a simple front end. The idea is to eventually let commiters request and configure their own experimental runs.
Jenkins – https://jenkins.freebsd.org. Uses bhyve VMs. Testing ports on all platforms.
Do not use freebsd-version(1) in the ports tree. Designed only for use in the base system. Security fixes that don’t touch the kernel won’t affect uname -r, and freebsd-version doesn’t apply to releases built from source. Ports tree needs something to say exactly what version you have no matter which how that version was produced.
Discussion on handling port licenses.
Packaging base! pkg doesn’t handle chflags yet, but they’re working on it. Split packages per build system option. But this changes how some programs are linked–what about NIS? Bapt is pondering that. We could offer multiple versions of packages, such as NIS-free. But FreeBSD’s “build system is not a paragon of configurability, but a bunch of hacks on what annoyed people the most” (Warner Losh).
I’m teaching in less than an hour, so I left the discussion here.
Some notes from the FreeBSD virtualization devsummit. Very rough, but my understanding is very rough, so all is as it should be.
Bhyve moving to UEFI loader away from FreeBSD and grub2
• Fork of intel EDKII (BSD License), OVMF build target
• For bhyve instead of Qemu
• Includes CSM BIOS emulation for non-EFI aware OS’s
• Currently in-house, being moved to public git repo
• Buildable on FreeBSD (GCC 4.6 or later), needs to be a port – bhyve folks need port creation help
• Serial console only: working on VGA emulation with VNC client
Networking:
• Virtio doesn’t support modern networking features
• One NIC, e1000 ((multiq, jumbo frames, TSO) under way
• e1000e (82580) dev emulation in progress
• each has thousands of registers, still working on them
Considering:
• user mode using WANProxy/libuinet
• simple kernel eth switch
Storage
• zvol GEOM-avoidance in place (mav@) – prevent geom from sniffing ZFS partition tables, so host will never see VM filesystem
• virtio todo: asynch block writes, add virtio SCSI
• Wanted: BSD-licensed sparse image tools for working with vmdk, qcow2, vhd, etc. Would be nice to point bhyve at a VMDK file and so “go!”
Future
• AMD-SVM
• Windows guest support (requires UEFI)
• Illumos doesn’t need UEFI, needs a real BIOS – use BIOS compat in UEFI
• ARM(64) chips have virtualization support, get bhyve to work on it.
• Save state/restore/migrate
• configuration file, as the command line is unwieldy for hierarchical info – use UCL because the ports people also use UCL
• Regression suite – bhyve supports lots of different hardware and operating systems, so we need to have automated testing
Other virtualization
• Virtualbox – FreeBSD is tracking very closely, 4.3.10 came out 25 March, port updated on 28 March.
• HyperV – 10.0, amd64 and i386 guest support
o Recent Azure image announcement
o Nobody in the FreeBSD community tracks Hyper-V, it could use a nanny
Luigi Rizzo on performance with device drivers
• One option – e1000 emulation, performance will be poor, will be slow
• Some emulation drivers fake TSO, etc
• No good solutions outside paravirtualization
• High performance = modify guest device driver to be virtualization-aware
• Luigi got 17GB/s using netmap with bhyve
Roger Pau Monne’ on Xen
• Changes in FreeBSD 10
o Vector callback for injecting event channel interrupts
o PV timer
o PV IPIs
o Add Xen support into GENERIC – can now use freebsd-update
o Sponsored by SpectraLogic and Citrix
• PVH domU
o Supported guest mode since 4.4
o Builds atop of the PVHVM work introduced in FreeBSD 10
o Half-merged into –current, some work remains
o Same speed as PVHVM, main difference is way it boots
o Not as intrusive as a traditional PV port
• PVM Dom0
o Xen side patches almost fully merged
o Main difference between PVH DomU and Dom0 is that on Dom0 FreeBSD needs to manage the hardware
o Add support for PIRQ (physical interrupts routed atop event channels)
o ACPI tables parsed by Dom0, and Xen must be made aware of the underlying devices
o Xen user-space control devices needed by the toolstack:
Privcmd – allows issuing hypercalls into Xen and mapping foreign domain memory from userspace
Evtchn – allows registering and receiving interrupts by user-space applications
• Big items remaining
o Add multiboot support to the FreeBSD bootloader – right now, you must use pxelinux or grub
o Improve if.xn – doesn’t work correctly with a NetBSD dom0, doesn’t work properly between guests on the same host, paravirtualized interface does not perform well yet.
• Hoping to have Xen work for FreeBSD 11
VirtIO/VMWare guest drivers by Brian Venteicher
• Work done over last year
• VirtIO: new
o Unmapped IO – block and SCSI
o Network multiqueue
o Random (entropy) device
o Initial console driver –can do multi-consoles, hotplug is so-so
• VirtIO: remaining
o Support missing devices – MMIO
o Non-x86 architectures
o SCSI multiqueue
o VirtIO version 1 specification – very similar to existing virtio
• VMWare
o Vmxnet3
Vmware provided driver, messy
OpenBSD imported their own vmx driver May 2013
Ported to FreeBSD 10.0
• TSO/LRO offload
• Multiqueue
To do: PVSCSI & VMWare tools
Device emulation in bhyve
• Most emulated in userspace usr.sbin/bhyve
• Kernel ones in vmm/io/ (PICs and timers)
• ISA-LPC – uart, rtc
• PCI
o Virtio
Block – storage
Net – tap
Rng – random entropy from /dev/random
o Ahci
o Pass-through
• Go through how virtio device drivers work. Interesting, lots of diagrams he should post, but way above my head, so I didn’t take too many notes
• Virtio random number generator
o Usr.sbin/bhyve/pci_virtio_rnd.c
o Guest rng driver requests 32-bit number to replenish its random pool
o FreeBSD /dev/random is non-blocking, using Yarrow and (soonish) Fortuna
I’m at the BSDCan FreeBSD devsummit, and the current topic is FreeBSD 11 Goals.
As the Great Committer John Baldwin has requested that people take notes and blog about the discussions, and this might be of wider interest, here’s the goals.
These are my notes. I probably missed things. I would be shocked if I didn’t, actually. And I probably misunderstood some stuff.
Test suite/QA (jmmv) – some stuff merged to 10
Mips64 & more MIPS stuff
Scatter/Gather mbufs (scottl) – collapse down mbufs from a long chain into one unit
Lldb (emaste) – make it first-class citizen, fully functioning & working in 11 on all platforms, native cross-platform debugging
Uefi boot and install support (emaste)
Package the base system (gjb/bapt)
Open to floor:
AES GCM added to ipsec – jmg
ASLR – Shawn “The Goats Are His Fault” Webb
DNS improvements – Erwin
Suspend/resume
Libc++
OpenMP
FreeBSD devs – Want icc? Talk to gnn@
Kload – hot swap kernel upgrade
Dragonfly mailer as default
Ncurses cleanup
Capsicum and casper improvements – use casper to help apps use dnssec correctly
TCP performance and enhancements (gnn) – project as a whole needs broader TCP patch reviewers
L2 rework
Libuinet
Arm64 (andyt)
Package building Mips32 packages via qemu
External toolchain improvements (imp) – some people need GCC
Remove gcc by 11 – gnn willing to remove it right after this devsummit session
Remove ia64 (marcel)
Useland dtrace (marki)
Xen dom0 for x86 (rogier)
Kqueue64 from osX – available from Apple, we could pull this in (jmg)
Async sendfile (glebius)
Lightweight reference counts (maybe) (glebius)
Kdbus (need for desktop)
Vt + newcons default (emaste)
KMS, DRM, AGI impromements (dumbbell & kip)
SMT (need)
Encrypted kernel dump (gnn)
Nand flash (warner)
Superpages for certain arm & mips
Multi-endian ufs
Libdispatch (sson)
Move libraries to private (need, bapt, bdrewery help)
/etc/src.conf improvements
64-bit linuxulator
newer linuxulator (xmj)
new autofs (emaste)
unionfs improvements (need)
64-bit struct stat & dirent, mount_pathlen, max_pathlen, (benno)
X32 – alternate abi for amd64 – jhb wants, but no commitment
PF – improve its internal API so we can manage stable branches, merge newer version, IPv6 improvements (glebius)
Bhyve – UEFI
Vxlan
Reproducible release builds – remove usernames, host names, timestamps from builds
IPv6 security improvements
Network stack backpressure
Network multipath (stretch goal)
Capsicum shell – will sandbox package building
Non-root image building
Will all this happen? Who knows. But plans are nice.
BSDCan is next week. I will be there. You should too.
This year’s conference includes goats. For reasons too complicated and sordid for me to explain. If you really want to know, see here.
Goat BOF. Tuesday night. Royal Oak pub. Be there or be not one of the cool kids.
And as an afternote: I feel really sorry for Bryan Lunduke.
“Hey, where is Lucas? Why hasn’t he posted lately?”
I’ve done nothing worth posting about. Most of this month I spent removing a per-millennial switch from the core of the network, which was painstaking and annoying but not noteworthy. I then spent nine days at a writing workshop, which was fascinating, educational, and utterly exhausting. I could argue that the workshop was worth blogging about, but I was too busy writing to waste time writing. If you’re interested in writing, though, and you have a chance to do any of Dean or Kris’ workshops, go.
So:
Next weekend, I’ll be at Penguicon, appearing on various panels. You can see me at the following one-hour events.
Friday
Saturday
Sunday
Now if you’ll excuse me, I have a whole great big heap of slides to do…
I hadn’t planned to post this, but enough people asked me that I feel obliged to explicitly state:
DetroitBSDCon is a joke. So is Oracle buying BSDCan. I did not play off of Dan’s posting: we planned it together, as well as the resulting fight on Twitter. (I must concede that Dan won the Twitter argument by enlisting Randi Harper for Oracle BSDCan. Nobody can stand against @freebsdgirl‘s awesome social networking mojo. Mind you, Dan has absolutely no clue about how we do things here in Detroit.)
I don’t expect anyone to believe anything posted anywhere on 1 April. Dan and I did not expect to fool anyone, but we did find the idea funny. And so did a lot of other people, so that’s okay. A few folks hate 1 April in general, but they’re not going to change the world. I won’t do gag posts on random days–unless, of course, something is laugh-so-hard-you’ll-herniate-yourself funny and must go on a certain day as part of the joke.
I’ve done three 1 April gags: this one, the Great Committer in 2011, and FretBSD (also with Dan) in 2003. I only do them if my inspirational muse kicks me in the head.
A surprising number of people contacted me about DetroitBSDCon — not because they believed it, but because they want me to do it. They don’t care if I hold it in an abandoned factory, they just want DetroitBSDCon to happen. I have run conferences before, but these days I lack the time, energy, and flexibility to do so. Plus, it fails the WIBBOW test. Like, utterly fails the WIBBOW test. Fails with screeching and tears and thrashing about on the ground, running from the test room bawling like a whipped piglet.
Holding a conference is easy. A lot of work, but it’s very straightforward work.
If you want a BSD event in your city, here’s what you do.
My conference appearances for this year end in May. I don’t want to travel. But if you have a BSD event within a couple hours drive of Detroit, and it didn’t conflict with my prior commitments, I’d show up. (Or, if you ask politely, I’ll stay home. Whatever you prefer.)
Dan Langille has sold BSDCan to Oracle. From the early announcement, it’s clear that they’ll ruin the conference. I take this VERY personally, as I’ve worked with BSDCan for over a decade. Dan has made it clear that he’s taking the check and walking away without a second thought. This is unconscionable.
If I want something done about it, I’ll have to do it myself.
OpenBSD committer Nick Holland lives about two miles from me. We’ve had some discussions about what needs to happen to give the Western Hemisphere a truly free and unencumbered BSD conference. With Dan’s acceptance of Oracle’s offer, we’ve been forced to put these plans into action. As Nick has no real Internet presence, I’ve been elected to announce our efforts on my blog.
Coming in May 2014: DetroitBSDCon!
Detroit is a major transportation hub, with a well-connected airport and one of the world’s busiest border crossings. People will have no trouble getting here.
Having a conference in Detroit gives us interesting possibilities, however. Traditional conference space is limited, and very hard to get at such late notice. Fortunately, the BSD community is very open to non-traditional conferences.
One of the disadvantages to holding a conference in May is that the weather is just starting to get nice. Most of us have been trapped inside all winter, and now that it’s getting warm enough to be outside we all crowd into a stuffy windowless room for presentations. DetroitBSDCon will be a little different. Allow me to present: the Packard Plant.
One square mile of abandoned industrial space, including offices, manufacturing floors, and more. It’s all the space we could possibly use. Each presentation or tutorial will get its own floor. Yes, some parts of the plant are deathtraps, but they’re fairly obvious.
Best of all, we get no end of fresh air. The surrounding area is nice and quiet.
There’s always a chance that the weather will not cooperate. The rental agency providing the chairs, tables, projection gear, and other assorted conference paraphernalia has agreed to throw in a bunch of propane pole heaters as part of the deal.
I work for an ISP, so Internet isn’t a problem. The whole conference will be wireless. Nick has kindly volunteered to climb the water tower and mount the kit for the gigabit wireless uplink.
Accommodations are actually very inexpensive. Detroit hosted the Super Bowl in 2006, and many people opened hotels just for that event. These days, you can get a room for free if you agree to a) not set it on fire, and b) cook meth only in the bathtub.
And dining? Yes, there aren’t many restaurants near the Packard Plant, but we have something better than boring old sit-down restaurants. As the economy has essentially collapsed, the more entrepreneurial folks have opened unofficial dining establishments. You’ll see things like this by every major road.
We’re arranging for dinner to come to you. Detroit has some of the world’s best barbeque and soul food, and it’ll all be there for you. Yes, smelling lunch and dinner cooking might be something of a distraction during the conference presentations, but let’s be real a moment: you go to the presentations to have a chance to work on your laptop in peace. Delicious aromas won’t hamper that in the slightest.
And beer? Another nice thing about living in a collapsed city is that people will deliver beer by the truckload anywhere you want at any time. For a modest extra fee at registration, you’ll get a wristband that gets you free beer throughout the conference. (Speakers get a boozeband for showing up.)
The dates for DetroitBSDCon are the same as those for Oracle BSDCan. Because seriously, how many BSDCan attendees are actually going to go to Oracle BSDCan?
Programming is the hardest and most important part of a conference, and there’s not much time to get papers together. We’ve decided to steal the entire BSDCan programming slate. Because, seriously, those guys aren’t going to want to talk for Oracle.
Speakers won’t need to change their travel arrangements, however. We’ve reserved cars on Canada’s Via Rail train system, leaving Ottawa on Tuesday, Wednesday, and Thursday nights, making the run down to Detroit. It’s Via Rail First class because, again, free booze. They’ll bring you to Windsor overnight, where you’ll hop the bus to the conference venue. We’ll put you up at some of the closest hotels, such as Hot Sheets Central, Scabies R Us, and Bedbugs Bonanza. Yes, they’re lower-end hotels, but seriously, after the University of Ottawa dorms, they’re fine. Plus, free beer.
The after-party will take place Saturday night, on a train back to Ottawa so speakers can catch their flights out the next day.
Now, some speakers might choose to go to Oracle BSDCan. They could. They have free will, after all, and they’re free to make their own decisions even if they’re wrong. In the event we have open spots in the program, Nick and I will fill in with various BSD-related presentations we’ve given over our many years in the BSD communities. We’ve found slides for talks like “Removing IPF from OpenBSD” and “ATAng: Supporting ATA Drives into the 21st Century,” so we’re all set to shore up weak spots in the program.
Best of all, Nick and I promise to never sell DetroitBSDCon. To Oracle.
See you in the ruins in May!