Monday, October 31, 2011

What's cooking for FreeBSD 9?

This page will document changes that will be included in FreeBSD 9, including those that might end up being committed to earlier branches. In other words, it describes differences between 8.0 and 9.0, no matter what happens to the versions in between.
For information on the currently released 8-STABLE branch see What's cooking for FreeBSD 8 page.
Some of the more important low-level changes can be seen in the future release's UPDATING file.
Also useful are the quarterly Status Reports:
Everyone is encouraged to download a snapshot CD image and try all the new features (as well as the old ones). Developers are very interested in bug reports. Note that FreeBSD 9.0 is not released yet and both the snapshots and the default source trees have debugging enabled by default (which results in dramatic slowdowns so don't benchmark them without removing the debugging options).
If you're interested in how FreeBSD gets developed, you're encouraged to read the mailing lists and developer blogs.

Overall system / architectural changes

Userland DTrace

Status: committed to -CURRENT.
Will appear in 9.0: sure
Author: Rui Paulo
Web: announcement, commit message
The kernel parts of the DTrace system diagnostic framework were imported some time ago, but they are now completed with the support for userland tracing, making it usable in general userland software development and system administration. Userland DTrace is already used in some large well known software packages such as PostgreSQL and X.Org.

CLANG / LLVM compiler

Status: Committed to -CURRENT.
Will appear in 9.0: yes
Authors: Roman Divacky, Pawel Worach, Ed Schouten and many others
Web: LLVM, CLANG, FreeBSD wiki, mailing list announcement
As the GCC compiler suite was relicensed under GPLv3 after the 4.2 release, and the GPLv3 is a big dissapointment for some users of BSD systems (mostly commercial users who have no-gplv3-beyond-company-doors policy), having an alternative, non-GPL3 compiler for the base system has become highly desireable. Currently, the overall consensus is that GCC 4.3 will not be imported into the base system (the same goes for other GPLv3 code).
The LLVM and CLANG projects together offer a full BSD-licenesed C/C++ compiler infrastructure that is, performance and feature-wise close to, or better than GCC. The LLVM is the backend and the CLANG is the front-end part of the infrastructure.
Recent development has shown that not only is it possible to start using LLVM+CLANG right away, it is also very stable. The probability of replacing GCC for the base system in the near future is high. LLVM/CLANG will also add benefits to the overall system such as better error reporting, Apple's Grand Central Dispatch system for developing multithreaded applications and possibly JIT compiling some internal structures like firewall rules.
Note that this mostly affects the base system. There is too much third party software (e.g. ports) that depends on GCC to completely replace it.
Update: To make this happen, PathScale has developed and donated the C++ runtime library under the BSD license. PathScale has some other tools and libraries which may in the future help the transition to a BSD-licensed toolkit: the assembler (pathas) and the debugger (pathDB).

Kernel & low level improvements

Large-scale SMP support

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: Atillio Rao and others
Web: commit message
This work brings in support for large SMP systems, with more than 32 CPUs. Previously, the kernel structures were unable to account for such a large number of CPUs so the newest method implements extensible CPU accounting. This is not an improvement in scalability in itself but is a prerequisite for large-scale SMP work.

USB 3.0 support

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: Hans Petter Selasky
Web: commit message
The new wave of USB changes improves on the hugely successful USB development released in FreeBSD 8 and brings in support for USB 3.0.

Network kernel core dumps (netdump)

Status: In development
Will appear in 9.0: probably
Author: Attilio Rao and others
Web: announcement
Netdump is a framework that aims for handling kernel coredumps over the TCP/IP suite in order to dump to a separate machine than the running one. That may be used on an interesting number of cases involving disk-less workstations, disk driver debugging or embedded devices.

Initial NUMA support

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: John Baldwin
Web: Commit message, discussion
As NUMA-like architectures have become almost ubiqutous, even in i386 / amd64 architectures, there are potentially big performance gains to be had in enabling its supports within operating systems. New development aims to adapt the physical page allocator to be NUMA-aware.

Modern event timer infrastructure

Status: Committed to -CURRENT
Will appear in 9.0: sure
Authors: Alexander Motin
Web: Commit message 1 Commit message 2
To better support the many sources of timer ticks present in todays system and to build the foundation for tickless kernel, a new unifying timer infrastructure was created. It currently supports LAPIC, HPETs, i8254, RTC.

Tickless kernel

Status: Under development
Will appear in 9.0: probably
Authors:Tsuyoshi Ozawa, Alexander Motin
Web: idea, new timer infrastructure
To improve performance in virtual machines and power usage in laptops, the "dynamic tick mode" (also called, a bit inappropriately, "tickless mode") can replace the classic, strictly periodic hardware timer interrupt ticking with one-shot variable-time ticks. This will save some CPU time which would otherwise be spent handling timer interrupts which have no work assigned to them.

Networking improvements

More SMP-scalable TCP/IP

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: Robert Watson
Web: commit message announcement
Improvements to the networking stack introduce better scalability strategies based on thework by Alan Cox and others. With these changes, it is expected that the connections will have more clear CPU affinity, less cache line contention and better use of modern hardware flow detection and handling.

New NFS client and server

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: Rick Macklem
Web: commit message
The new NFS client and server introduce the support for NFSv4 as their biggest features, with ACL support, byte range locking and delegation support. It should also be easier to maintain and later upgrate do NFSv4.1

Five new TCP congestion algorithms

Status: Partially committed to -CURRENT
Will appear in 9.0: sure
Author: Lawrence Stewart
Web: Commit message
This commit marks the first formal contribution of the "Five New TCP Congestion ontrol Algorithms for FreeBSD" FreeBSD Foundation funded project. More details about the project are available at: http://caia.swin.edu.au/freebsd/5cc/.

SIFTR - Statistical Information for TCP Research

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: Lawrence Stewart
Web: commit message
SIFTR logs a range of statistics on active TCP connections to a log file, providing the ability to make highly granular measurements of TCP connection state. The tool is aimed at system administrators, developers and researchers.

Storage subsystems' improvements

A move to support 4K drives

Status: committed
Will appear in 9.0: sure
Authors: Kirk McKusick, Alexandar Motin, Andrey V. Elsukov and others
FreeBSD's GEOM and file systems have intrinsically supported large (or even arbitrary) sector sizes for a long time, but there is still the issue of detecting them and communicating this information across the layers. Some new development introduced SATA quirks to detect known 4K drives (with the ability for users to set their own quirks on non-detected drives), the gpart(8) utility will calculate the correct alignment or warn on misalignment, and the default fragment / block size for UFS was changed to 4K / 32K.

Generic GEOM IO schedulers

Status: Committed to -CURRENT
Will appear in 9.0: sure
Authors: Luigi Rizzo, Fabio Checconi
Web: commit message
The new framework, integrated with GEOM, allows for multiple disk IO schedulers to be used, if necessary, on different IO providers (e.g. drives). The usage of some IO schedulers can increase responsiveness in certain kinds of IO workloads, for example a mix of sequential and random IO.

HAST - High Availability Storage

Status: Committed to -CURRENT, planned MFC
Will appear in 9:0: sure
Author: Pawel Jakub Dawidek
Web: FreeBSD wiki page
HAST is a userland-based (ggate) implementation of a distributed storage device concept, similar to Linux's DRBD. It allows over-the-network mirroring of any GEOM storage devices in a semi-synchronous way (writes suceed when the data is sent over the wire).

UFS SoftUpdates+Journal (SU+J)

Status: Pending commit to -CURRENT
Will appear in 9.0: sure
Author: Jeff Roberson
Web: discussion
A new feature added to existing UFS SoftUpdates code makes use of a small journal, technically an intent log, to keep track of metadata garbage collection which has upto now been left as a job for (background) fsck after an unclean shutdown. The intent behind this is to eliminate the requirement for fsck or background fsck on file systems with SoftUpdates enabled after unclean shutdown.
In effect, this feature combines the best of both worlds - the very fast operation of SoftUpdates with the removal of the need for fsck characteristic for journalling file systems. This is not a radical change - the well known SoftUpdates mechanism is still in its original form - but it completes the garbage collection step in a different way.

New driver for AHCI SATA drives

Status: Committed to -CURRENT, MFC-ed
Will appear in 9.0: sure
Author: Alexandar Motin & others
Web: commit message
The new driver supports native AHCI via the CAM (common access method for storage) subsystem. AHCI drives are manipulated by camcontrol and support for new features like NCQ and port multipliers has been integrated. Among other features, performance has been significantly increased, port multipliers and hot-plugging are greatly improved.

ATA CAM implementation

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: Alexandar Motin
Web: commit message
The ATA disk drivers have all been moved to the CAM system, improving some features of them along the way. This makes CAM a very real central point and foundation of disk interfaces and management of (S)ATA, SCSI, USB and Firewire drives. Some SCSI controllers still have drivers outside CAM.
Improvements include: transfer size increase, better support for port multipliers.

Security

Capsicum

Status: Basic functions committed to -CURRENT
Will appear in 9.0: sure
Author: Robert Watson and others
Web: commit message
Capsicum is a framework for security isolation of sensitive processes, which may prove useful in security strengthening the operating system.

AES-XTS encryption mode in kernel

Status: Committed to -CURRENT
Will appear in 9.0: sure
Author: Pawel Jakub Dawidek
Web: AES-XTS in kernel AES-XTS in GELI AES-XTS via AESNI
The XTS block cypher mode is specially suited for encrypting disk drives and other block devices. It avoids some security problems arising with using plain CBC chaining with addressible-sector encryption.
AES with XTS mode is used in GELI and is also supported when implemented via the AES-NI.

NFSv4 ACLs for UFS

Status: Committed to -CURRENT, MFCed
Will appear in 9.0: sure
Author: Edward Tomasz Napierala
Web: commit message
The well known and loved UFS file system has for some time implemented POSIX.1e ACLs(access control lists) in addition to the classic Unix file permissions model. This file permission model greatly enhances the way files can be managed and allows new security models to be implemented. It is also a standard part of the FreeBSD kernel, ready to be used at any time.
However, the POSIX.1e standard apparently never became trully widespread in practice. Through market share domination (but not completly without technical merit) the NTFS (Microsoft Windows file system) ACL security model has become widely popular and implemented, even so that it directly inspired the ACL model in the NFS (Network File System) version 4. The POSIX model is simpler and more Unix-like but the NTFS/NFSv4 model is more expressive.
The two ACL models are incompatible - security parameters set in NFSv4 model cannot always be directly translated to the POSIX model. Due to this and considering that NFSv4 ACLs are already directly implemented in ZFS, the introduction of NFSv4 ACLs in UFS is simply a feature-completness step which makes both file systems similarily usable from NFSv4 clients.
The POSIX model still remains in the implementation, but is mutualy exclusive (at the mount-point level) with the NFSv4 model.

Other changes

The following is a list of smaller and / or more obscure changes that nevertheless deserve a special mention since they will be of interest to certain users:
As always, all features described here are, or will be, a part of the FreeBSD "base" system, available in every FreeBSD installation without patching or out-of-the-ordinary configuration.
For more information about development of FreeBSD (among other topics), see my blogwith daily and miscellaneous information.

Reference:
http://ivoras.net/freebsd/freebsd9.html
Posted by Jun Hsieh at 11:36 AM
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)