This year, BSDCan passed a critical threshold: we got too many good proposals. As a conference, this is an excellent problem to have. As a concom member, it kind of sucks. When your papers are mostly excellent, how do you decide which proposals to take and which to reject?
What follows is my highly personal take on BSDCan proposals and the evaluation procedure. It should not be taken as BSDCan concom gospel.
Generally speaking, a person who gets persuaded to present at BSDCan submits in later years. This is great–it means we don’t have to track them down again. It means we’re doing a good job at running an educational, friendly, fun, worthwhile event. Repeat attendees and speakers mean you’re doing something right.
In some previous years, we’ve had difficulty filling the conference. In prior years the convention committee did a lot of speaker hunting and trapping. So let’s talk about the committee.
When BSDCan started the BSD projects were fairly small. The handful of us could track what was happening everywhere. Over the last decade the BSD projects expanded. We needed people who keep up on the innards of these projects, and who could give perspective on papers from the project’s perspective. We have George Neville-Neil for FreeBSD, and Bob Beck for OpenBSD. We have Dru Lavigne, because she not only knows bloody everyone, she knows if people are decent speakers or not. David Maxwell originally gave us our NetBSD insight. He’s somewhat moved on from NetBSD, but he still works hard for the BSDCan committee, so we’re not about to let him go. The concom could use a leading NetBSD person to both judge NetBSD papers, and to help drag the NetBSD community into proposing more.
Dan Langille is the chairman, but most of his work is in the area of keeping the BSDCan servers and sites running, arranging physical space, and such. While he evaluates papers, he clearly expects the concom to make decisions on what gets included and what doesn’t.
There’s also this Lucas dude. I have no idea why they keep him around, as he’s far too self-centered to contribute much of any use.
Another part of each concom member’s role is to advocate for projects and proposals they believe in. I expect Bob to stand up for interesting OpenBSD papers. I expect George to pimp interesting FreeBSD papers. I expect both of them to be able to rank proposals from their posse, so that we can have a useful give-and-take and reach the best possible combination of presentations.
So how do we evaluate proposals? We use Pentabarf, which lets each concom member rate proposals, contacts submitters, and helps organize the schedule.
But the concom has to evaluate the papers. I’ve put some criteria that we use, in random order. Others apply them differently, or have additional criteria, but I think none of them actively disagree with mine.
There’s more to it than this, of course. I dug around trying to get more women to submit. I wouldn’t upvote a proposal because of the submitter’s gender, but I want to at least see them submit. After my efforts to let women know they’re welcome to submit, I was really pleased to see two very good kernel-related proposals from women. Having to say “we want women, but not you” would have really sucked.
So, after all that, let’s talk about proposals.
Having too many good proposals means that the average proposal quality increased. Proposals that would have been accepted in previous years based solely on the speaker’s reputation suddenly didn’t look so good next to the others.
I expect the proposals to continue to improve in the future. I’m writing this blog post so that some of our usual suspects have a useful model for writing good proposals.
Ideally, I would present both accepted and rejected talks here, and talk about why each met their fate. There’s problems with that, though. BSDCan doesn’t say up front that rejected talks will be made public. Organizations like USENIX say “we will do anything we want with your proposal,” but BSDCan doesn’t do that. (Whether we should or not is a separate discussion, and one the concom will have.) I went so far as to ask several rejected contributors if I could use their rejected proposals as examples. I’ve decided to not do that in this article, however. For one, it’s already too long, and I really need to generate some words on the ZFS book today. For another, negative criticism is both easy and less useful. Anyone can kick a dead dog. It’s much more useful to say “This was a great proposal, and here’s why.”
We have two separate sets of events: tutorials and presentations. On the tutorial side, we’ve learned the hard way that sysadmin tutorials attract more attendees than networking talks. The farther you go from a Unix command line, the fewer BSDCan people are interested. The attendance only supports 2 days of tutorials, in one room. This year, we got twice that many tutorial proposals. If you want to submit a tutorial, a half-day one is much more likely to fly than a full day one. We took Peter’s PF tutorial, because it’s well attended every year. Sysadmins do DNS, and DNSSEC is absolutely booming in some parts of the world. Despite my passionate argument against the doofus doing the FreeBSD storage tutorial, they took it. And Luigi’s netmap proposal was well done.
So, what’s in a good proposal? Here’s a couple examples I really liked. The details are not yet posted, but these will be public soon.
First, here’s “Expanding RDMA (Remote Direct Memory Access) capability over Ethernet in FreeBSD” by Shany Michaely. Mrs. Michaely has never presented at BSDCan before.
RDMA (Remote Direct Memory Access) is growing in popularity in Linux and Windows systems as a way to transfer large amounts of data with low latency and minimal involvement from the CPU. However RDMA InfiniBand drivers in FreeBSD were not updated, requiring users to create or port their own implementation of RDMA, and RDMA over Ethernet was not available in FreeBSD. This talk will describe how RDMA works and review the new addition of RoCE (RDMA over Converged Ethernet) network drivers in FreeBSD, allowing easier implementation of rapid data transfers with low CPU utilization over Ethernet and InfiniBand. This also enables the use of iSCSI over RDMA via the iSER (iSCSI Extensions for RDMA) protocol.
full description:
One of InfiniBand’s valuable capabilities is its support for RDMA (Remote Direct Memory Access) operations across a network, which enable rapid data transfer without involvement of the host CPU in the data path, and data placement to the responder memory without requiring its CPU awareness.
RoCE (RDMA over Converged Ethernet) is a standard for RDMA over Ethernet.
It provides true RDMA semantics for Ethernet and allows InfiniBand transport applications to work over an Ethernet network.
FreeBSD is frequently used for storage purposes and RDMA capability has a high potential of improving performance in such storage applications.
A good example for that is iSER (iSCSI Extensions for RDMA), a module being developed nowadays for FreeBSD, which enables the use of iSCSI over RoCE.
The main idea of this talk is a short overview of RDMA – Its principles, key components and its main advantages. Additionally, it will cover the use of RoCE – implementation architecture, obstacles we overcame in the development, and a quick browse of RoCE’s different capabilities and milestones.
So, what’s in this?
For a second example, here’s Matt Ahren’s proposal for “New OpenZFS features supporting remote replication.” Again, this will be public soon. Mr. Ahrens is a BSD conference veteran.
OpenZFS send and receive forms the core of remote replication products, allowing incremental changes between snapshots to be serialized and transmitted to remote systems. In the past year, we have implemented several new features and performance enhancements to ZFS send/receive, which I will describe in this talk.
Full description:
This talk will cover:
– Resumable ZFS send/receive, which allows send/receive to pick up where it left off after a failed receive (e.g. due to network outage or machine reboot).
– ZFS receive prefetch, which is especially helpful with objects that are updated by random writes (e.g. databases or zvols/VMDKs).
– ZFS send “rebase”, which can send changes between arbitrary snapshots; the incremental source is not restricted to being an ancestor of the snapshot being sent.
In this talk, I will cover the impact of these changes to users of ZFS send/receive, including how to integrate them into remote replication products. I will also give an overview of how zfs send/receive works, and how these enhancements fit into the ZFS codebase.
What do we have here?
So, here’s two good proposals.
While I don’t want to talk about specific rejected papers, I will touch on some mistakes people made.
One annoyance I had was people giving us multiple proposals. Don’t get me wrong–multiple proposals are good. They let us pick the talk that fits best with the conference. But the people who write good proposals sent multiple good proposals. They’re a pain to pick between!
I’m going to name Peter Hessler here, because he said I could.
Peter offered a wickedly excellent tutorial proposal on OpenBSD routing daemons, plus two good OpenBSD talks. Routing daemons are interesting to me, but BSDCan has learned that the further you get from the system, the fewer people attend the talk. We turned him down. That disappointed me, because I would have heckledattended. I hereby officially encourage Peter to submit that tutorial to a more network-related conference.
Of his talk proposals, we chose the most highly scored in Pentabarf. Now, Bob Beck is the OpenBSD rep on the concom. If he had said “No, we really want his other talk,” we’d probably go along with it. That’s a part of why Bob is on the concom.
We don’t accept BSDCan talks solely on the proposal grade in Pentabarf. If we had just said “Accept the top 40 talks, as rated by the concom in Pentabarf,” this would have been a very different conference, and many people would have been extremely unhappy with the results. But a strong proposal with a lot of information makes it easy for the concom to rate your proposal highly. If you’re willing to spend four hours making slides for your talk, spend a little time making a solid proposal for those slides.
I recommend creating an outline for your slides, and using that outline as the basis of your proposal. That’s what I do.
I can also say that the committee will be having discussions on how to cope with the “too many good talks” problem. If we’re lucky, this problem will only grow worse every year. I look forward to the day that we get so many excellent talks that we have to tell Kirk McKusick that we’re turning him down.
This post has gone on far longer than I intended. Thank you for reading this far, if you did. I hope that some of you who were rejected this year try again in 2016 and beyond, with proposals that make me say “oh, hell yes, we need this talk!”
“Thank you for reading this far, if yuo (sic) did.”
Well, at least you can spell floccinaucinihilipilification :-). See you in June!
Thanks, Adam! Fixed.
And yes, I spelled the important words correctly. I’ve always wanted an excuse to use that in a book…
Just throwing in my $0.02 on the issue of diversity. I am speaking for myself, no one else.
It’s a messy situation that can easily be rectified, at least in the future, if people are looking for diverse conference content with the projects all represented. I will “name names” in regards to the projects.
The fact is that FreeBSD people always have more submissions. There are more of them. That’s a fact. But in order to even it up, BOTH the con organizers AND the projects need to be a little more proactive on it.
To the con organizers: some general sense of quotas should be implemented. Don’t have enough meetings from DFly? Hit their lists, tell them, spam Matt D., get Justin S. to post. Take the initiative. Of course you know your con will be better if attendees get to hear about LibreSSL. Cross-pollination is often the best product of cons. Pawel from FreeBSD talking to Matt from DFly talking about file systems at NYCBSDCon in 2008 was a great moment to many of us in NYC.
To the projects: you know what matters from your current dev work. Harass your devs to speak on all the things that the rest of the community needs to hear about. Cons are a great platform. Take OpenBSD, besides LibreSSL, how about the OpenNTPD stuff which almost competes with OpenSSH in terms of being BSD software in Linux distros. All the ARM stuff happening. pf can always be a meeting. Status of 10g networking. Shall I continue?
NYC*BUG doesn’t always have the projects well-represented. We know it’s a constant battle to keep diversity in NYCBSDCon and in our monthly meeting content. There are more people with FreeBSD commit bits in NYC than any other BSD by far. But we take that as a challenge to implement diversity regardless.
And I will attending BSDCan, if only in order to hear MWL pronounce that make-believe word.
g