FreeBSD 9.0 — Fast, Safe and Completely Geeky

Most of the development that took place in this cycle had to do with bringing ZFS in FreeBSD in sync with the latest available open version (which is ZFS v28), improving the efficiency of FreeBSD Jails, adding support for USB 3.0 and making it all a lot faster.

Here’s what they did to the internal end of things:

UFS has gained journalling support for soft-updates.

UFS introduced soft-updates as a method for updating file system metadata (stuff like last accessed time, and permissions) out of band. This basically means that the data and the metadata don’t get committed to disk at the same time. While it’s a boon for performance, it’s nightmare for data integrity. With FreeBSD 9, they’ve added in an intent-log to the soft-updates mechanism, so soft-updates is now fully journalled. So on the outside, what you’ll see is that after a power failure, FreeBSD 9 now does not need to run fsck on the disks at all – it just comes back up online like nothing ever happened!

FreeBSD 9 has TRIM support for SSDs.

SSD users need to minimise writes to them. The ATA specification specifies a command called TRIM, which is used by operating systems to tell SSDs that a certain sector isn’t being used anymore by the file system. This alleviates the need to overwrite the sector to delete data. Linux users with SSDs will be familiar with the “discard” flag that they put in the /etc/fstab files that enables TRIM support. Well, FreeBSD 9 now supports TRIM.

ZFS has been updated to version 28.

ZFS v28 now supports block-level deduplication, so if you’ve got many files which have only some common segments, those segments can be stored only once by ZFS at the file-system level. Admittedly, deduplication requires a lot of RAM — the ARC cache itself takes over 20GB of RAM (yes, 20!) — so you’ll want a box with 48 or so gigs of memory to really take advantage of this. Triple-parity RAIDZ support is in. Also, ZFS v28 supports changing the log device (if you’re using a separate log device, that is,) so if your log device dies, your entire file system won’t. Like UFS, ZFS is also TRIM aware. And finally, ZFS now supports diffing between two files systems.

There have been improvements to low-level disk management.

Apart from improvements to RAID, HAST (Highly Available Storage) is now available as part of the system. HAST is just like Linux’s DRBD (which basically lets you transparently mirror a block device across a network), and supports only two nodes in an active-passive configuration.

Networking has been overhauled.

A bunch of network card drivers have been completely rewritten or majorly overhauled. In addition, TCP/IP now supports pluggable congestion-control algorithms, and FreeBSD comes with five of them in the box to choose from. So, for example, you can use one type of algorithm if you expect a moderate number of high bandwidth connections, and another if you expect a high number of low bandwidth connections. Of course, the default algorithm is balanced for regular use.

There’s a new storage subsystem.

FreeBSD now has a storage subsystem which implements the entire SCSI command set. So you get to take advantage of things like NCQ and all (stuff like sending a bunch of commands to the disk at once and letting the disk decide the order in which to execute them for best performance) because the disks are now used in their native mode and not ATA emulation mode. GEOM can now use multiple IO schedulers simultaneously, which is a really neat feature because it allows you to optimize disk IO even if your disk activity is erratic at times and sustained at other times. Of course, only one scheduler can be used on one drive. Also, Advanced Format (AF) drives with 4K sectors are now natively supported, and finally, you can now enable or disable the write cache either globally or per drive. This is neat because you might want to enable it on ZFS because ZFS ensures data is consistent so you can use the extra performance caching offers, but if you’re using a conventional file system, you’ll want to keep the write cache disabled for critical data.

There’s a new Resource Usage Monitoring And Limiting API

FreeBSD can now monitor resource usage per user, per jail, or per monitoring class. Also, the groundwork for controlling resources have been laid out, so in the future versions, something called rcontrol will be able to limit usage of resources per user, per jail and per monitoring class too. This makes for an interesting capability — you can limit jails to CPU sets right now, and in the future you will be able to limit available RAM to jails too, so in essence you’re getting everything that VMs can do on jails but there’s no performance penalty.

FreeBSD doesn’t crash anymore if you remove a USB drive.

Eh? Well, with previous versions of FreeBSD, there was a bug where if you just yanked a USB drive out without unmounting it, there was a 50 per cent chance that the system would kernel panic. The bug stayed on until they rewrote the entire USB subsystem. The new USB subsystem now fully supports USB 3.0 XHCI, and fixes the kernel panic issue for good.

There are upgrades to SSH and resov.conf handling.

The latest OpenSSH has been imported from OpenBSD, and in addition to that, the OpenSSH-HPN patches (HPN is High Performance Networking) have been imported too, and makes OpenSSH a lot faster in some scenarios. Also, FreeBSD 9 now uses OpenResolv, which means that resolv.conf files can be configured per application, per networking device or per connection (using a different one for Wi-Fi and VPN). Multiple resolv.conf files can be active at the same time, and OpenResolv hooks in at the library level, so the system /etc/resolv.conf can stay unaffected.

Security has been improved.

Password hashing now supports SHA256 and SHA512 in addition to MD5 and Blowfish. Also, there’s a new framework called Capsicum, developed at the University of Cambridge, that allows you to sandbox individual applications. Capsicum is still in-the-works stuff, so you can’t use all of it just yet.

They’re moving away from GPL

Because of GPLv3 and its implications, FreeBSD is now moving away from dependencies on all GPL tools. The system still compiles with GCC, but that’s GCC 4.2 because that was the last version that was GPLv2 based. They’ve already imported in Apple and University of Illinois at Urbana-Champaign’s LLVM framework and Clang, and that should be the system compiler from FreeBSD 10 onwards. Apart from the fact that they are licensed much more liberally, LLVM and Clang have technical advantages too — they generate code that are orders of magnitude faster, and somewhat smaller.

Well, that’s for all the internal changes that have gone in this cycle. The shell has also been updated, and they’ve updated handling for arithmetic expressions and environment variables for builtin commands, and imports code from Debian’s Dash shell, which in turn imports code from NetBSD’s Ash shell, which is why the code was BSD licensed in the first place.

One of the biggest changes that the user will see (or not — I always install my BSD systems by manually extracting the tarballs and creating config files by hand) is that FreeBSD has a new installer program called BSDinstall. Sysinstall was cool — and it’s still available for configuring the system — but all ISOs from FreeBSD now ship with BSDinstall as the default installer.

Over at the PC-BSD stable, they’ve done a few major changes too. If you’ve used PC-BSD, you’ll know that they use a package manager somewhat like Mac OS X, where they install PBI packages which are self-contained — they include all their dependencies in them. Well, PC-BSD packages can now share libraries too! And most importantly, PC-BSD packages gained support for package signing, so now the integrity of a package can finally be verified.

Also, PC-BSD is no more KDE-exclusive. You can now use GNOME, XFCE and even LXDE with PC-BSD. Which makes me wonder — the GhostBSD project was started to provide GNOME to FreeBSD. Do they have a niche anymore, now that PC-BSD makes GNOME a first class citizen?

What’s not so rosy at Free/PC-BSD is the state of graphics. Mesa has really abandoned development for almost all non-Linux platforms because all their drivers are now based on Gallium3D — which is neat and Gallium3D drivers give some really snazzy performance, but Gallium3D depends on the DRM (Direct Rendering Manager) which is as of now Linux only.

Mesa has removed all of their classic DRI based drivers, so you can either use an old version of Mesa, use the new LLVMpipe-based software renderer (which can do Gnome-shell or Compiz pretty well), or buy a nVidia GPU and use their proprietary driver (which, according to Phoronix, is the only way you’re going to enjoy FreeBSD 9 with an acceptable level of graphics performance). But for typical server workloads (which is what FreeBSD is for, hey), this isn’t much of an issue as no one’s likely to use OpenGL on a server.

That’s pretty much all, folks. Oh, yeah, FreeBSD can now do binary updates using binary diffs, so you don’t need to fetch source from CVS and do a make world just to keep your system up to date.

I’ve talked enough. Now go play with this.