Blog

I finished the OpenZFS Mastery chapter on volumes earlier this week.

Zvols support sparse volumes, also known as thin provisioning or overcommitting storage. A sparse volume is a volume that reserves less space than the volume actually needs. Sparse volumes permit allocating more space than the pool has available. You could, for example, create ten 1-exabyte sparse volumes on your 1 TiB dataset. So long as your volumes are never heavily used, nobody will notice that you’re overcommitted. Sparse volumes can be useful if you’re prepared to add more disks. Using a sparse volume means you can tell the Windows host that a disk is a specific large size, even if the underlying pool doesn’t have that much free space.

Sparse volumes are not generally recommended. If the pool fills up but the sparse volumes think they have space remaining, writes will fail with an “out of space” error. Guests will get errors they can’t hope to resolve. Windows in particular has low distress tolerance.

I’d appreciate your support keeping the rats fed as I finish writing this book.

I’m trying to finish up the OpenZFS Mastery chapter on disk space use today. Well, my part of it. I’m planning to dump–delegate, delegate–the section on deduplication to Allan. When you have a co-author expert, use them.

Zstd offers compression levels from 1 to 19. The default is zstd-3, a reasonable compromise between performance and efficiency. Writing data at higher compression levels require more CPU but might save more space. Zstd has a useful quirk where decompressing higher levels uses no more processing power than decompressing lower levels. If you have data that must be written once but read many times, increasing the compression level makes sense.

ZFS still supports gzip compression, but Zstd is always faster and achieves better results while using less processor and memory.

ZFS also offers the ZLE algorithm, for data with long runs of zeroes. Some file formats, such as GIFs and certain scientific data, include long runs of zeroes. If you have a dataset for such data, you might find ZLE compression useful.

How does one realistically tell if your data has long runs of zeroes? The simplest way I know is to pass it across a T1 with 23 channels set to b8zs but one channel stuck on AMI. Such a circuit cannot pass large all-zero packets. Sadly, this recommendation does not suit a book on filesystems.

I would appreciate your help keeping the lights on as I finish this book.

I’m writing about space usage OpenZFS Mastery.

ZFS offers tools to transparently increase the amount of data you can fit on a disk. Rather than requiring the system administrator to compress and decompress individual files, ZFS uses efficient compression at the filesystem level. While compressing logs during rotation makes the files look smaller in ls(1), it doesn’t change how many disk blocks those logs use. Snapshots and clones let you duplicate entire datasets with only a tiny amount of space. ZFS can also perform deduplication of files, improving disk usage at the cost of memory. We’ll see how to evaluate these options and determine when to use each.

If ZFS is so transparent, why is space usage complicated? Child datasets affect their parents. Those space-efficient clones can grow, and snapshots can prevent freeing space.

I’d appreciate your help keeping the lights on while I finish this book.

Probably jinxing myself by posting this, but: here’s a chunk from Twisted Presents.

Three nights ago had been the Longest Dark, the strongest night of the year. A night for the clan to come together and declare the year’s struggles, to mark the end of feuds and wars. Even living clanless and bereft, Mha had caught a whisper that her youngest daughter was in danger and had charged into battle. The magnificent fight had given Mha fresh cuts. She had defeated an elven wizard. Other orcs had honored her. December knew the truth, though. A daughter claimed by another clan no longer belonged to her mother, but she’d fought for her anyway. She’d had a flimsy cowhide of an excuse, but excuses weighed nothing against ice.

Soon she’d return to the barn. Back to her coal-fed stove and the sepulchral groom’s quarters where she rotted her days. For now, she bore December’s castigation.

Mha’s night-sight made the farmhouse’s lingering warmth a red rectangle. Humans had a holiday a few days after the Longest Dark, a holiday that Thorn’s neglectful nephew could not ignore. Thorn would return tomorrow but tonight, with nobody to feed the furnace, the snug farmhouse cooled.

Want to know when this will hit the world? Watch the Kickstarter.

OpenZFS Mastery is starting to pick up momentum. Don’t look at it too closely. You might scare it.

First, note how ridiculously wide this VDEV is. A fourteen-disk VDEV will have suboptimal performance and experience slow death by fragmentation. Also, the devices were named badly. Now that drives have failed, it matters. But we’re not here to gawk at others’ past mistakes—uh, learning experiences. We’re looking at hardware failures.

This RAID-Z2 pool is in a degraded state. It’s missing not one but two disks. One of the drives showed a red light and was removed. The other failed silently. A RAID-Z2 VDEV can handle up to two disk failures. This pool has lost all redundancy. One more disk failure will destroy the pool. A pool without redundancy cannot self-heal, so if a file suffers bit rot, it will not be corrected. This pool is very vulnerable.

I’d appreciate your support while writing this silly thing.

OpenZFS Mastery is starting to roll lunch again. Hoping to get some intertia with it.

But create a dataset for each team, and give each site its own dataset within that parent dataset, and possibilities multiply. A team needs a copy of a web site for testing? Clone it. With traditional filesystems, you’d have to copy the whole site directory, doubling the space used and taking much, much longer. Clones use only the amount of space for the differences between the datasets and can be created instantaneously. When the team is about to deploy a new version of a site but wants a backup of the old site, they can create a snapshot. This new site probably uses a whole bunch of the same files as the old one, so you’ll reduce disk space usage. When the deployment inevitably goes horribly wrong, they can restore the old version by reverting to the snapshot. A particular web site needs filesystem-level performance tweaks, or compression, or some locally created property? Set it for that site. When you must change a ZFS property on all of the sites, make the change to the parent dataset and let the children inherit it.

OpenZFS Mastery is yet again open for sponsorships. Just like yesterday, and last month, and seemingly forever.