This year, BSDCan passed a critical threshold: we got too many good proposals. As a conference, this is an excellent problem to have. As a concom member, it kind of sucks. When your papers are mostly excellent, how do you decide which proposals to take and which to reject?
What follows is my highly personal take on BSDCan proposals and the evaluation procedure. It should not be taken as BSDCan concom gospel.
Generally speaking, a person who gets persuaded to present at BSDCan submits in later years. This is great–it means we don’t have to track them down again. It means we’re doing a good job at running an educational, friendly, fun, worthwhile event. Repeat attendees and speakers mean you’re doing something right.
In some previous years, we’ve had difficulty filling the conference. In prior years the convention committee did a lot of speaker hunting and trapping. So let’s talk about the committee.
When BSDCan started the BSD projects were fairly small. The handful of us could track what was happening everywhere. Over the last decade the BSD projects expanded. We needed people who keep up on the innards of these projects, and who could give perspective on papers from the project’s perspective. We have George Neville-Neil for FreeBSD, and Bob Beck for OpenBSD. We have Dru Lavigne, because she not only knows bloody everyone, she knows if people are decent speakers or not. David Maxwell originally gave us our NetBSD insight. He’s somewhat moved on from NetBSD, but he still works hard for the BSDCan committee, so we’re not about to let him go. The concom could use a leading NetBSD person to both judge NetBSD papers, and to help drag the NetBSD community into proposing more.
Dan Langille is the chairman, but most of his work is in the area of keeping the BSDCan servers and sites running, arranging physical space, and such. While he evaluates papers, he clearly expects the concom to make decisions on what gets included and what doesn’t.
There’s also this Lucas dude. I have no idea why they keep him around, as he’s far too self-centered to contribute much of any use.
Another part of each concom member’s role is to advocate for projects and proposals they believe in. I expect Bob to stand up for interesting OpenBSD papers. I expect George to pimp interesting FreeBSD papers. I expect both of them to be able to rank proposals from their posse, so that we can have a useful give-and-take and reach the best possible combination of presentations.
So how do we evaluate proposals? We use Pentabarf, which lets each concom member rate proposals, contacts submitters, and helps organize the schedule.
But the concom has to evaluate the papers. I’ve put some criteria that we use, in random order. Others apply them differently, or have additional criteria, but I think none of them actively disagree with mine.
We must evaluate proposals as they are given. We want to accept the best proposals. We know how well our repeat presenters speak, but we still have to look at what they send us. What are they going to talk about? What are they going to cover? How technical is this talk? How much detail is in the proposal? Is this well thought out?
When we have lots of proposals, one talk per presenter. I could easily fill two days of talking on various technical things, but that would turn into “Lucas is a Nutter Con.” Nobody wants to see that, and besides, I do that at Penguicon. (Penguicon has a whole hotel, though, so they just shove me in a room and let me get on with it.) One of the reasons that people come to BSD cons is to hang out with BSD developers, experts, and assorted related characters. The more of those we can get into the con, the better experience people will have.
I’m biased towards bringing new people into BSDCan. I’ll happily go see Henning Brauer, Kirk McKusick, or George Rosamond speak. The first two are technical, while George is as enthusiastic as a puppy who’s just discovered peanut butter. But I don’t want to go to a con that has the same people year after year. The BSD community is growing. BSDCan needs to grow with it.
Concom members can submit, but we recuse ourselves from voting on our own proposals. To eliminate ourselves would be foolish–for example, Bob Beck’s fantastic LibreSSL talk would not have happened if concom could not present. But we let the other concom members decide if we’re going to talk. And being a member of the concom is not a guarantee of acceptance. We rejected David Maxwell’s proposal this year, and he hasn’t stormed off the committee in disgust or anything.
Tracks. We want some sysadmin, some kernel, some hacking, some “I don’t know what this is but it sounds dang cool.”
Presenter. Why should you be the one to give this talk?
Originality. Has this talk been presented before? We don’t demand originality, but if a talk has appeared before that’s a mark against it.
New. Is this new tech? New processes? New software? And yes, there’s a difference between “here’s my new prototype that nobody but me has ever seen” and “here’s new, but in production in several environments” software. Your previously-unseen basement prototype had better be spectacular if you want us to take your talk.
Draw. BSDCan is a business, albeit a nonprofit one. We either break even or die. Everything costs more than you budget, which means that if you plan to break even you’ll lose money. Having “big names” means that people are more likely to attend, which means more registration fees and more sponsors.
Can you speak? We’ve attended talks that look great on paper but have an unintelligible speaker. Your reputation will precede you. If you can’t speak clearly and distinctly, we won’t ask you back. Learn to speak in public. If you have this problem, but you’ve done something to remediate it–say, Toastmasters, or the Dale Carnegie public speaking courses, or something–say so.
BSD diversity. I want many different BSDs at BSDCan.
There’s more to it than this, of course. I dug around trying to get more women to submit. I wouldn’t upvote a proposal because of the submitter’s gender, but I want to at least see them submit. After my efforts to let women know they’re welcome to submit, I was really pleased to see two very good kernel-related proposals from women. Having to say “we want women, but not you” would have really sucked.
So, after all that, let’s talk about proposals.
Having too many good proposals means that the average proposal quality increased. Proposals that would have been accepted in previous years based solely on the speaker’s reputation suddenly didn’t look so good next to the others.
I expect the proposals to continue to improve in the future. I’m writing this blog post so that some of our usual suspects have a useful model for writing good proposals.
Ideally, I would present both accepted and rejected talks here, and talk about why each met their fate. There’s problems with that, though. BSDCan doesn’t say up front that rejected talks will be made public. Organizations like USENIX say “we will do anything we want with your proposal,” but BSDCan doesn’t do that. (Whether we should or not is a separate discussion, and one the concom will have.) I went so far as to ask several rejected contributors if I could use their rejected proposals as examples. I’ve decided to not do that in this article, however. For one, it’s already too long, and I really need to generate some words on the ZFS book today. For another, negative criticism is both easy and less useful. Anyone can kick a dead dog. It’s much more useful to say “This was a great proposal, and here’s why.”
We have two separate sets of events: tutorials and presentations. On the tutorial side, we’ve learned the hard way that sysadmin tutorials attract more attendees than networking talks. The farther you go from a Unix command line, the fewer BSDCan people are interested. The attendance only supports 2 days of tutorials, in one room. This year, we got twice that many tutorial proposals. If you want to submit a tutorial, a half-day one is much more likely to fly than a full day one. We took Peter’s PF tutorial, because it’s well attended every year. Sysadmins do DNS, and DNSSEC is absolutely booming in some parts of the world. Despite my passionate argument against the doofus doing the FreeBSD storage tutorial, they took it. And Luigi’s netmap proposal was well done.
So, what’s in a good proposal? Here’s a couple examples I really liked. The details are not yet posted, but these will be public soon.
First, here’s “Expanding RDMA (Remote Direct Memory Access) capability over Ethernet in FreeBSD” by Shany Michaely. Mrs. Michaely has never presented at BSDCan before.
RDMA (Remote Direct Memory Access) is growing in popularity in Linux and Windows systems as a way to transfer large amounts of data with low latency and minimal involvement from the CPU. However RDMA InfiniBand drivers in FreeBSD were not updated, requiring users to create or port their own implementation of RDMA, and RDMA over Ethernet was not available in FreeBSD. This talk will describe how RDMA works and review the new addition of RoCE (RDMA over Converged Ethernet) network drivers in FreeBSD, allowing easier implementation of rapid data transfers with low CPU utilization over Ethernet and InfiniBand. This also enables the use of iSCSI over RDMA via the iSER (iSCSI Extensions for RDMA) protocol.
full description:
One of InfiniBand’s valuable capabilities is its support for RDMA (Remote Direct Memory Access) operations across a network, which enable rapid data transfer without involvement of the host CPU in the data path, and data placement to the responder memory without requiring its CPU awareness.
RoCE (RDMA over Converged Ethernet) is a standard for RDMA over Ethernet.
It provides true RDMA semantics for Ethernet and allows InfiniBand transport applications to work over an Ethernet network.
FreeBSD is frequently used for storage purposes and RDMA capability has a high potential of improving performance in such storage applications.
A good example for that is iSER (iSCSI Extensions for RDMA), a module being developed nowadays for FreeBSD, which enables the use of iSCSI over RoCE.
The main idea of this talk is a short overview of RDMA – Its principles, key components and its main advantages. Additionally, it will cover the use of RoCE – implementation architecture, obstacles we overcame in the development, and a quick browse of RoCE’s different capabilities and milestones.
So, what’s in this?
This proposal clearly states its BSD connection, including which BSD it uses.
This proposal discusses implementing a new technology
This proposal says “we accomplished X. This is how. This went wrong. This is what we learned. Beware of these dragons.”
For a second example, here’s Matt Ahren’s proposal for “New OpenZFS features supporting remote replication.” Again, this will be public soon. Mr. Ahrens is a BSD conference veteran.
OpenZFS send and receive forms the core of remote replication products, allowing incremental changes between snapshots to be serialized and transmitted to remote systems. In the past year, we have implemented several new features and performance enhancements to ZFS send/receive, which I will describe in this talk.
Full description:
This talk will cover:
– Resumable ZFS send/receive, which allows send/receive to pick up where it left off after a failed receive (e.g. due to network outage or machine reboot).
– ZFS receive prefetch, which is especially helpful with objects that are updated by random writes (e.g. databases or zvols/VMDKs).
– ZFS send “rebase”, which can send changes between arbitrary snapshots; the incremental source is not restricted to being an ancestor of the snapshot being sent.
In this talk, I will cover the impact of these changes to users of ZFS send/receive, including how to integrate them into remote replication products. I will also give an overview of how zfs send/receive works, and how these enhancements fit into the ZFS codebase.
What do we have here?
Detail on what will be discussed and newly implemented features. Anyone can look at this proposal and say “there is meat on this bone.”
ZFS is a draw, and will bring people into the con. That helps.
The proposal includes things like the impact of these changes and how these new features fit into the software.
So, here’s two good proposals.
While I don’t want to talk about specific rejected papers, I will touch on some mistakes people made.
One-sentence descriptions. We judge the talk’s merit based on what you tell us. Anyone can say “I will talk about foobar.” You’re going up against proposals like the two above. Give us detail. The concom cannot read your mind. I had no idea whatsoever RDMA over Ethernet was a thing before reading that proposal. Every fooBSD developer might know your name and work–but I don’t. Once upon a time I knew everyone, but that time is years past. And I’m rating your proposal.
Not being BSD-specific. We had a very nice proposal for a talk on a popular scripting language. The proposal did not mention BSD. If we have a surplus of talks, cutting non-BSD talks from a BSD con is the obvious route.
Not including BSD at all. We had a couple proposals from sysadmins who work in really interesting environments, where simply keeping hardware and people alive is a challenge. I truly adore those talks. But, if your proposal doesn’t mention which BSD you use, for all we know you’re running everything on Windows XP. Again, this is a BSD conference.
Secrecy. Every few years we get a proposal that says “Here’s my provocative talk title. I’m not telling you anything about the talk.” Uh, no. That gives the concom no basis to make a decision on, and we’re responsible for the content of the conference. (It also gives attendees no basis to decide if they want to see your talk, which is less of an issue depending on who the speaker is.)
Spelling and grammar are not pimple cream, to be dabbed on where needed. If you’re going to indulge in floccinaucinihilipilification, you’d better be able to spell it.
Speaking of floccinaucinihilipilification: don’t submit a proposal that consists of running down other people’s work. We’re not interested. You’re better off saying why your stuff is awesome as opposed to denigrating someone else.
One annoyance I had was people giving us multiple proposals. Don’t get me wrong–multiple proposals are good. They let us pick the talk that fits best with the conference. But the people who write good proposals sent multiple good proposals. They’re a pain to pick between!
I’m going to name Peter Hessler here, because he said I could.
Peter offered a wickedly excellent tutorial proposal on OpenBSD routing daemons, plus two good OpenBSD talks. Routing daemons are interesting to me, but BSDCan has learned that the further you get from the system, the fewer people attend the talk. We turned him down. That disappointed me, because I would have heckledattended. I hereby officially encourage Peter to submit that tutorial to a more network-related conference.
Of his talk proposals, we chose the most highly scored in Pentabarf. Now, Bob Beck is the OpenBSD rep on the concom. If he had said “No, we really want his other talk,” we’d probably go along with it. That’s a part of why Bob is on the concom.
We don’t accept BSDCan talks solely on the proposal grade in Pentabarf. If we had just said “Accept the top 40 talks, as rated by the concom in Pentabarf,” this would have been a very different conference, and many people would have been extremely unhappy with the results. But a strong proposal with a lot of information makes it easy for the concom to rate your proposal highly. If you’re willing to spend four hours making slides for your talk, spend a little time making a solid proposal for those slides.
I recommend creating an outline for your slides, and using that outline as the basis of your proposal. That’s what I do.
I can also say that the committee will be having discussions on how to cope with the “too many good talks” problem. If we’re lucky, this problem will only grow worse every year. I look forward to the day that we get so many excellent talks that we have to tell Kirk McKusick that we’re turning him down.
This post has gone on far longer than I intended. Thank you for reading this far, if you did. I hope that some of you who were rejected this year try again in 2016 and beyond, with proposals that make me say “oh, hell yes, we need this talk!”
