Posted on

BSDCan proposals and rejections

This year, BSDCan passed a critical threshold: we got too many good proposals. As a conference, this is an excellent problem to have. As a concom member, it kind of sucks. When your papers are mostly excellent, how do you decide which proposals to take and which to reject?

What follows is my highly personal take on BSDCan proposals and the evaluation procedure. It should not be taken as BSDCan concom gospel.

Generally speaking, a person who gets persuaded to present at BSDCan submits in later years. This is great–it means we don’t have to track them down again. It means we’re doing a good job at running an educational, friendly, fun, worthwhile event. Repeat attendees and speakers mean you’re doing something right.

In some previous years, we’ve had difficulty filling the conference. In prior years the convention committee did a lot of speaker hunting and trapping. So let’s talk about the committee.

When BSDCan started the BSD projects were fairly small. The handful of us could track what was happening everywhere. Over the last decade the BSD projects expanded. We needed people who keep up on the innards of these projects, and who could give perspective on papers from the project’s perspective. We have George Neville-Neil for FreeBSD, and Bob Beck for OpenBSD. We have Dru Lavigne, because she not only knows bloody everyone, she knows if people are decent speakers or not. David Maxwell originally gave us our NetBSD insight. He’s somewhat moved on from NetBSD, but he still works hard for the BSDCan committee, so we’re not about to let him go. The concom could use a leading NetBSD person to both judge NetBSD papers, and to help drag the NetBSD community into proposing more.

Dan Langille is the chairman, but most of his work is in the area of keeping the BSDCan servers and sites running, arranging physical space, and such. While he evaluates papers, he clearly expects the concom to make decisions on what gets included and what doesn’t.

There’s also this Lucas dude. I have no idea why they keep him around, as he’s far too self-centered to contribute much of any use.

Another part of each concom member’s role is to advocate for projects and proposals they believe in. I expect Bob to stand up for interesting OpenBSD papers. I expect George to pimp interesting FreeBSD papers. I expect both of them to be able to rank proposals from their posse, so that we can have a useful give-and-take and reach the best possible combination of presentations.

So how do we evaluate proposals? We use Pentabarf, which lets each concom member rate proposals, contacts submitters, and helps organize the schedule.

But the concom has to evaluate the papers. I’ve put some criteria that we use, in random order. Others apply them differently, or have additional criteria, but I think none of them actively disagree with mine.

  • We must evaluate proposals as they are given. We want to accept the best proposals. We know how well our repeat presenters speak, but we still have to look at what they send us. What are they going to talk about? What are they going to cover? How technical is this talk? How much detail is in the proposal? Is this well thought out?
  • When we have lots of proposals, one talk per presenter. I could easily fill two days of talking on various technical things, but that would turn into “Lucas is a Nutter Con.” Nobody wants to see that, and besides, I do that at Penguicon. (Penguicon has a whole hotel, though, so they just shove me in a room and let me get on with it.) One of the reasons that people come to BSD cons is to hang out with BSD developers, experts, and assorted related characters. The more of those we can get into the con, the better experience people will have.
  • I’m biased towards bringing new people into BSDCan. I’ll happily go see Henning Brauer, Kirk McKusick, or George Rosamond speak. The first two are technical, while George is as enthusiastic as a puppy who’s just discovered peanut butter. But I don’t want to go to a con that has the same people year after year. The BSD community is growing. BSDCan needs to grow with it.
  • Concom members can submit, but we recuse ourselves from voting on our own proposals. To eliminate ourselves would be foolish–for example, Bob Beck’s fantastic LibreSSL talk would not have happened if concom could not present. But we let the other concom members decide if we’re going to talk. And being a member of the concom is not a guarantee of acceptance. We rejected David Maxwell’s proposal this year, and he hasn’t stormed off the committee in disgust or anything.
  • Tracks. We want some sysadmin, some kernel, some hacking, some “I don’t know what this is but it sounds dang cool.”
  • Presenter. Why should you be the one to give this talk?
  • Originality. Has this talk been presented before? We don’t demand originality, but if a talk has appeared before that’s a mark against it.
  • New. Is this new tech? New processes? New software? And yes, there’s a difference between “here’s my new prototype that nobody but me has ever seen” and “here’s new, but in production in several environments” software. Your previously-unseen basement prototype had better be spectacular if you want us to take your talk.
  • Draw. BSDCan is a business, albeit a nonprofit one. We either break even or die. Everything costs more than you budget, which means that if you plan to break even you’ll lose money. Having “big names” means that people are more likely to attend, which means more registration fees and more sponsors.
  • Can you speak? We’ve attended talks that look great on paper but have an unintelligible speaker. Your reputation will precede you. If you can’t speak clearly and distinctly, we won’t ask you back. Learn to speak in public. If you have this problem, but you’ve done something to remediate it–say, Toastmasters, or the Dale Carnegie public speaking courses, or something–say so.
  • BSD diversity. I want many different BSDs at BSDCan.

    There’s more to it than this, of course. I dug around trying to get more women to submit. I wouldn’t upvote a proposal because of the submitter’s gender, but I want to at least see them submit. After my efforts to let women know they’re welcome to submit, I was really pleased to see two very good kernel-related proposals from women. Having to say “we want women, but not you” would have really sucked.

    So, after all that, let’s talk about proposals.

    Having too many good proposals means that the average proposal quality increased. Proposals that would have been accepted in previous years based solely on the speaker’s reputation suddenly didn’t look so good next to the others.

    I expect the proposals to continue to improve in the future. I’m writing this blog post so that some of our usual suspects have a useful model for writing good proposals.

    Ideally, I would present both accepted and rejected talks here, and talk about why each met their fate. There’s problems with that, though. BSDCan doesn’t say up front that rejected talks will be made public. Organizations like USENIX say “we will do anything we want with your proposal,” but BSDCan doesn’t do that. (Whether we should or not is a separate discussion, and one the concom will have.) I went so far as to ask several rejected contributors if I could use their rejected proposals as examples. I’ve decided to not do that in this article, however. For one, it’s already too long, and I really need to generate some words on the ZFS book today. For another, negative criticism is both easy and less useful. Anyone can kick a dead dog. It’s much more useful to say “This was a great proposal, and here’s why.”

    We have two separate sets of events: tutorials and presentations. On the tutorial side, we’ve learned the hard way that sysadmin tutorials attract more attendees than networking talks. The farther you go from a Unix command line, the fewer BSDCan people are interested. The attendance only supports 2 days of tutorials, in one room. This year, we got twice that many tutorial proposals. If you want to submit a tutorial, a half-day one is much more likely to fly than a full day one. We took Peter’s PF tutorial, because it’s well attended every year. Sysadmins do DNS, and DNSSEC is absolutely booming in some parts of the world. Despite my passionate argument against the doofus doing the FreeBSD storage tutorial, they took it. And Luigi’s netmap proposal was well done.

    So, what’s in a good proposal? Here’s a couple examples I really liked. The details are not yet posted, but these will be public soon.

    First, here’s “Expanding RDMA (Remote Direct Memory Access) capability over Ethernet in FreeBSD” by Shany Michaely. Mrs. Michaely has never presented at BSDCan before.

    RDMA (Remote Direct Memory Access) is growing in popularity in Linux and Windows systems as a way to transfer large amounts of data with low latency and minimal involvement from the CPU. However RDMA InfiniBand drivers in FreeBSD were not updated, requiring users to create or port their own implementation of RDMA, and RDMA over Ethernet was not available in FreeBSD. This talk will describe how RDMA works and review the new addition of RoCE (RDMA over Converged Ethernet) network drivers in FreeBSD, allowing easier implementation of rapid data transfers with low CPU utilization over Ethernet and InfiniBand. This also enables the use of iSCSI over RDMA via the iSER (iSCSI Extensions for RDMA) protocol.

    full description:

    One of InfiniBand’s valuable capabilities is its support for RDMA (Remote Direct Memory Access) operations across a network, which enable rapid data transfer without involvement of the host CPU in the data path, and data placement to the responder memory without requiring its CPU awareness.

    RoCE (RDMA over Converged Ethernet) is a standard for RDMA over Ethernet.

    It provides true RDMA semantics for Ethernet and allows InfiniBand transport applications to work over an Ethernet network.

    FreeBSD is frequently used for storage purposes and RDMA capability has a high potential of improving performance in such storage applications.

    A good example for that is iSER (iSCSI Extensions for RDMA), a module being developed nowadays for FreeBSD, which enables the use of iSCSI over RoCE.

    The main idea of this talk is a short overview of RDMA – Its principles, key components and its main advantages. Additionally, it will cover the use of RoCE – implementation architecture, obstacles we overcame in the development, and a quick browse of RoCE’s different capabilities and milestones.

    So, what’s in this?

  • This proposal clearly states its BSD connection, including which BSD it uses.
  • This proposal discusses implementing a new technology
  • This proposal says “we accomplished X. This is how. This went wrong. This is what we learned. Beware of these dragons.”

    For a second example, here’s Matt Ahren’s proposal for “New OpenZFS features supporting remote replication.” Again, this will be public soon. Mr. Ahrens is a BSD conference veteran.

    OpenZFS send and receive forms the core of remote replication products, allowing incremental changes between snapshots to be serialized and transmitted to remote systems. In the past year, we have implemented several new features and performance enhancements to ZFS send/receive, which I will describe in this talk.

    Full description:

    This talk will cover:
    – Resumable ZFS send/receive, which allows send/receive to pick up where it left off after a failed receive (e.g. due to network outage or machine reboot).
    – ZFS receive prefetch, which is especially helpful with objects that are updated by random writes (e.g. databases or zvols/VMDKs).
    – ZFS send “rebase”, which can send changes between arbitrary snapshots; the incremental source is not restricted to being an ancestor of the snapshot being sent.

    In this talk, I will cover the impact of these changes to users of ZFS send/receive, including how to integrate them into remote replication products. I will also give an overview of how zfs send/receive works, and how these enhancements fit into the ZFS codebase.

    What do we have here?

  • Detail on what will be discussed and newly implemented features. Anyone can look at this proposal and say “there is meat on this bone.”
  • ZFS is a draw, and will bring people into the con. That helps.
  • The proposal includes things like the impact of these changes and how these new features fit into the software.

    So, here’s two good proposals.

    While I don’t want to talk about specific rejected papers, I will touch on some mistakes people made.

  • One-sentence descriptions. We judge the talk’s merit based on what you tell us. Anyone can say “I will talk about foobar.” You’re going up against proposals like the two above. Give us detail. The concom cannot read your mind. I had no idea whatsoever RDMA over Ethernet was a thing before reading that proposal. Every fooBSD developer might know your name and work–but I don’t. Once upon a time I knew everyone, but that time is years past. And I’m rating your proposal.
  • Not being BSD-specific. We had a very nice proposal for a talk on a popular scripting language. The proposal did not mention BSD. If we have a surplus of talks, cutting non-BSD talks from a BSD con is the obvious route.
  • Not including BSD at all. We had a couple proposals from sysadmins who work in really interesting environments, where simply keeping hardware and people alive is a challenge. I truly adore those talks. But, if your proposal doesn’t mention which BSD you use, for all we know you’re running everything on Windows XP. Again, this is a BSD conference.
  • Secrecy. Every few years we get a proposal that says “Here’s my provocative talk title. I’m not telling you anything about the talk.” Uh, no. That gives the concom no basis to make a decision on, and we’re responsible for the content of the conference. (It also gives attendees no basis to decide if they want to see your talk, which is less of an issue depending on who the speaker is.)
  • Spelling and grammar are not pimple cream, to be dabbed on where needed. If you’re going to indulge in floccinaucinihilipilification, you’d better be able to spell it.
  • Speaking of floccinaucinihilipilification: don’t submit a proposal that consists of running down other people’s work. We’re not interested. You’re better off saying why your stuff is awesome as opposed to denigrating someone else.

    One annoyance I had was people giving us multiple proposals. Don’t get me wrong–multiple proposals are good. They let us pick the talk that fits best with the conference. But the people who write good proposals sent multiple good proposals. They’re a pain to pick between!

    I’m going to name Peter Hessler here, because he said I could.

    Peter offered a wickedly excellent tutorial proposal on OpenBSD routing daemons, plus two good OpenBSD talks. Routing daemons are interesting to me, but BSDCan has learned that the further you get from the system, the fewer people attend the talk. We turned him down. That disappointed me, because I would have heckledattended. I hereby officially encourage Peter to submit that tutorial to a more network-related conference.

    Of his talk proposals, we chose the most highly scored in Pentabarf. Now, Bob Beck is the OpenBSD rep on the concom. If he had said “No, we really want his other talk,” we’d probably go along with it. That’s a part of why Bob is on the concom.

    We don’t accept BSDCan talks solely on the proposal grade in Pentabarf. If we had just said “Accept the top 40 talks, as rated by the concom in Pentabarf,” this would have been a very different conference, and many people would have been extremely unhappy with the results. But a strong proposal with a lot of information makes it easy for the concom to rate your proposal highly. If you’re willing to spend four hours making slides for your talk, spend a little time making a solid proposal for those slides.

    I recommend creating an outline for your slides, and using that outline as the basis of your proposal. That’s what I do.

    I can also say that the committee will be having discussions on how to cope with the “too many good talks” problem. If we’re lucky, this problem will only grow worse every year. I look forward to the day that we get so many excellent talks that we have to tell Kirk McKusick that we’re turning him down.

    This post has gone on far longer than I intended. Thank you for reading this far, if you did. I hope that some of you who were rejected this year try again in 2016 and beyond, with proposals that make me say “oh, hell yes, we need this talk!”

    • Posted on

    n4sa Print Sneaking Out

    The print version of Networking for Systems Administrators is starting to appear for purchase on Amazon’s web site.

  • n4sa US
  • n4sa UK

    Other Amazon sites should appear shortly.

    Amazon has not yet performed their usual discounting. Usually, if you order a brand-new book they’ll retroactively drop the price. But not always. I’d tell you to add it to your cart today and check tomorrow.

    Vendors like Barnes & Noble and Powells will pick it up in a week or two.

    • Posted on

    Testing Reddit Advertising for Nonfiction

    The biggest boost in book sales I’ve ever seen is from when the review of FMSE went on Reddit’s sysadmin board.

    Reddit has a self-serve advertising platform, with a minimum buy of $5 for 5000 impressions. I’m willing to risk $5 on an experiment.

    So I now have a sponsored link on Reddit, pimping Sudo Mastery. I would post a link, except you lot would click through and mess up my impressions count. If you’re reading my blog, I’m going to assume you are aware that the book exists. If you have not bought the book, me shouting BUY MY BOOK will not change your mind.

    I spent $13.75 to run this ad from 3 February to 1 March on /r/linux.

    I then put $7.14 to run this ad from 3 February to 05 March on /r/bsd.

    I investigated /r/sysadmin, but they didn’t have advertising space available.

    So far this morning, I have 137 impressions and 2 clicks. But it’s the first day.

    To break even on non-time expenses alone, I need to make $21. That’s not a huge hurdle. I did spend a couple hours on this, but most of that was figuring out how the advertising platform worked. (Hint: while the self-serv advertising platform defaults to two-day campaigns, running campaigns over a month or so gives you much more chance your campaign will be accepted. And I’m told that repeated exposure to an ad, over a long period, is more effective than a short sharp insistence.)

    I am curious any of my readers who are regular Redditors happen to see the ad in the course of their normal day, however.

    Posted on

    “Networking for Systems Administrators” on Kindle

    After more time than I expected, Networking for Systems Administrators is available on Kindle. I’ve uploaded it to Nook and Kobo as well, but they haven’t made it available yet.

    The Amazon links on the book page are affiliate links. If you buy through them, I make a little extra.

    I’m working on the print layout today. Print will be available ASAP.

    Once I have an approved print proof, I’ll make the book available on tiltedwindmillpress.com. It’ll be another week or so, probably.

    Thanks to everyone who helped to make this a better book.

    And I’d especially appreciate any reviews people might have to offer. I didn’t make the ebook available before release to see if it affected my visibility on Amazon. (Rumor claims that initial sales velocity impacts a book’s visibility.) According to those same rumor mills, the number of purchaser reviews in the first week disproportionately impact ongoing visibility.

    Posted on

    Word doc placed in Indesign gets numbered headings

    This has bit me more than once, and according to Google it affects nobody else on the whole Internet. Do I feel special or what?

    I write books in Microsoft Word, using Styles. Paragraph and character styles are essential for producing electronic and print books.

    At random times, when I import a Word doc into Indesign for print layout, numbers appear before the headings. These numbers do not appear in the Word document. Resolving this problem drives me near to madness.

    InDesign and Word both have features to number chapters and sections. The “place document” process somehow tickles one of them just right to activate this behavior.

    To get rid of it:

  • Go into the Word doc.
  • Modify the problem Style.
  • Select Format
  • Go to Numbering
  • Select a numbering scheme, and immediately select “no numbering.”
  • Save the doc

    You can now import without numbered headings mysteriously appearing.

    Recording this for my future reference will hopefully vaccinate me against ever having this problem again. That’s what disaster preparations are for, after all!

    • Posted on

    “Tarsnap Mastery” pre-pub reviewers wanted

    The first draft of “Tarsnap Mastery” is ready for pre-pub review.

    Colin has reviewed the manuscript, so I’m fairly sure that the technical stuff is correct. But Colin knows Tarsnap very well, so he read the book already understanding what I’m trying to teach.

    And in aggregate, y’all use Tarsnap much more than I possibly could. There’s thousands of you. You’ve probably figured out ways to abuse the service that I would have never thought of.

    If you’re interested in doing a pre-pub review, send me an email with the subject “Tarsnap pre-pub review.” Please give me a sentence or two saying:

  • how experienced you are as a sysadmin
  • how experienced you are at Tarsnap

    I can’t take everyone who volunteers, but I want to get a fair balance between different levels of sysadmin and Tarsnap experience.

    The book draft PDF is 16MB. Send your message from an email address that can accept a file that large. If the email bounces, I’ll assume that you didn’t read this far in the instructions, and hence don’t really want to review.

    Please send any comments in plain text. I don’t look at comments until the due date, so one big response is fine.

    I’ll need any comments back by 9 February 2015.

    If you want more detail on pre-pub reviews, look at my web site.

    • Posted on

    FreeBSD Jail Management Tools

    To design “FreeBSD Mastery: Jails” I need to look at the existing jail management tools. Jails have been around about fifteen years now, and FreeBSD has accumulated a whole bunch of wrappers and supporting tools. Many of these have wound up in the ports collection.

    Jails have evolved over the years. Some of these add-on tools are not useful for FreeBSD 9.1 and later.

    Here’s a few things I discovered in my research. I’m hoping that you lot will offer your own comments and help me decide which tools to cover in the book.

    It seems we have five major jail management toolkits.

  • ezjail – perhaps the best known jail management tool. Written entirely in shell.
  • qjail – Designed for managing lots of jails at the command line, based on templates. The examples use ipfilter, which is my third choice of FreeBSD firewall. Does not need ZFS.
  • iocage – supports resource limiting, thin provisioning, cloning, and either vimage or NAT from the host’s main IP.
  • jadm – Python-based jail command shell, uses a bridge interface. Can migrate jails between hosts. ZFS integration. Lets you set global settings for all jails, per-jail settings, jail groups, and so on.
  • cbsd – web-based management of jails. Supports HAST, migration, CARP, etc.

    The question for me is: which should I cover in the jails book? I’ll mention that all of them exist, but I can only give attention to one or two.

    CBSD seems an obvious choice. It integrates CARP and HAST and vimage and just about everything. Plus, people like web GUIs. It seems to be the giant ape of jail management tools.

    But I want to cover a command-line toolkit. Between ezjail, qjail, iocage, and jadm, I find myself leaning towards iocage.

    There’s some other jail-related software in the ports collection. Here’s those I plan to investigate and possibly include. I might find that their functionality is now included in mainline FreeBSD, however.

  • jps and jtop – external wrappers that add jail info to ps and top.
  • jkill – shuts down a running jail and all its processes from outside the jail. I don’t know that this is still needed, but the functionality is important.
  • bsnmp-jails – feed jail info into snmpd.

    Here are some jail-related ports I don’t plan to include, and why.

  • py-ploy_ezjail, bsdploy – ploy for jails. I don’t ploy.
  • py-ezjailremote – a python wrapper around ezjail. I don’t Python.
  • p5-BSD-Jail-Object – a Perl interface for jail management. I do Perl, but… no.
  • pkg_jail – build packages inside a jail. This looks like an old poudriere.
  • jailrc – improved startup/shutdown scripts for pre-9.1 jails. The key words here are “pre 9.1.”
  • pam_jail – drops the user into a jail upon successful login
  • jailme – a modified version of jexec with more sanity checking. Is setuid, lets normal users run jails.
  • jaildaemon – lets the jail talk to the host? I’m sure this solved a problem for someone, but not me.
  • jailctl – for FreeBSD 4.x and 5.x
  • jailaudit – portaudit for inside jails. I’d say this is superceded by pkg audit.
  • jail2 – advanced jail script. Uses /etc/jail.conf. I’m kinda, sorta sure that this or its descendant is the default FreeBSD 9.1 and later.
    • Posted on

    Jan 2015 Status & Next Projects

    Welcome to 21 January 2015. Here’s the news.

    I finished the first draft of Tarsnap Mastery yesterday. Today I went through it one more time, then shipped it off to Colin Percival for his comments on the last few chapters. Once I have his corrections, I’ll solicit reviewers.

    Networking for Systems Administrators is at the copyeditor. It’s due back Friday. I expected this book to be pretty easy, but the tech reviewers savaged it. The end result will be a much better book, but it still wasn’t much fun and took longer to repair than I expected. With any luck, though, I’ll be able to get the electronic version out before the end of January and print in mid-February.

    These two books are not available for pre-order through the Tilted Windmill Press web site. Books that I offer direct pre-orders on have done much more poorly than books without direct pre-orders. (Part of that might be topics, of course.) Other authors tell me that Amazon uses a book’s initial sales velocity to compute a book visibility to other buyers. More than 90% of my TWP book sales come from Amazon, so I care what happens on that platform.

    The sequel to Immortal Clay is rolling along.

    Now that Tarsnap Mastery and Networking for System Administrators are in a lull, I’ve started seriously pulling material together and filling in the outlines for my next FreeBSD Mastery books:

  • Jails (#fmjail)
  • ZFS (#fmzfs)
  • Specialty Filesystems (#fmspf, because #fmsf is taken)

    If you watched my most recent BSDNow interview, this is not a surprise.

    The books will assume you know what’s in FM: Storage Essentials, and there is a certain amount of interconnection between all three. For example, to use jails you should know about devfs and unionfs. The Jails book will include the incantations to perform the devfs and unionfs tasks needed for jails, but the explanations for them will be in the Specialty Filesystems book. Similarly, the Jails book will have ZFS rituals in it, but the ZFS book will have the knowledge behind those rituals.

    So, if you know some of these systems but not all you only need buy exactly what you need.

    I might split the Specialty Filesystems book into two parts, one for local filesystems and one for networked filesystems, depending on how long the book gets and the final content. My goal for these books is to make them about 30,000-40,000 words.[1] FMSE was 45,000 words, and N4SA is about that long. The sudo and Tarsnap books are closer to 30K, while SSH is right around 35K.

    I expect that once I finish them, I’ll have a bundle at Tilted Windmill Press much like the existing Security Bundle.

    When will these be finished? I really want to take the finished print books to BSDCan in June.

    [1] Why restrict book length? While the various ebook platforms do not restrict how long books can be, they do restrict how much I can charge for them. Most traditional publishers do not have that restriction. I must stagger roughly around that fuzzy intersection between “give good value,” “include what the reader needs,” and “can’t pay the mortgage.”

    Note that the on-demand printers do limit the size of print books. The bindings at the larger size books are not great. I refuse to release shoddy print books.