Considering FreeBSD Jails

FreeBSD has a long and strange history, tracing its roots back to the original Unix from Bell Labs. FreeBSD can almost be seen as the older brother to the younger, more popular Linux. While they are similar, FreeBSD’s long history has given it a very different philosophy and purpose. You can, and many people do, run FreeBSD as a desktop OS, but that’s not where the systems real talent lies. While Linux has tried, and in many cases succeeded, to be everywhere that a kernel can run, FreeBSD has decided that it belongs in the datacenter, a belief that is personified in the FreeBSD slogan, “The Power to Serve”. FreeBSD’s Jails are a great example of where the philosophy of Linux and FreeBSD diverges and produces systems that are functionally similar but logically very different.

In a traditional virtual machine, built the way VMware, Xen, and VirtualBox do VMs, the virtualization application runs an entire operating system as an independent entity. This requires the application to virtualize the hardware, producing CPU, RAM, and storage in software. The application then boots a new kernel in the virtualized environment, and runs specialized drivers, like VMware tools, inside of the newly booted operating system to allow communication between the two systems.

FreeBSD jails are very different. The jail does not boot its own kernel, and does not run a full version of the operating system. A jail is comprised of a filesystem, a hostname, an IP address, and an application. Jails can be seen as the logical successor to the older chroot environment, which restricted an applications access to the filesystem by providing the application it’s own root. Jails expand on this concept by further separating the host operating system and the application they are running. The difference between virtual machines and jails can be summed up by saying that virtual machines are for operating systems, jails are for applications.

Virtual machines have become popular in part because of the ease of deployment of a new operating system, spinning up a new operating system is as simple as clicking a button or two. However, from a systems administration point of view, once the new operating system is deployed, you then have a new operating system, as in, yet another box to manage. That means another box to keep patched and up to date, another box to keep user accounts on, another box to monitor in Nagios, in short, additional managerial overhead for each virtual machine. Not to mention administration of the host server. All of these detractors can be mitigated with additional tools like cfengine or Puppet, and creation of appropriate templates can make deployment and management easier.

Virtual machines have also become popular thanks to the dropping price and availability of amazingly powerful hardware. It is not at all uncommon for a single blade to ship with dual-six-core processors and 64G of RAM. To adequately utilize all of that power, virtualization lets you load up multiple operating systems into RAM. It has been my experience that RAM becomes the bottleneck in a virtualized environment far before CPU does.

I have a quote hanging by my desk at work that reads:

It is more important to reduce the Effort of Maintenance than it is to reduce the Effort of Implementation.

The Effort of Maintenance is proportional to the complexity of the system.
The quote is in reference to software design, but it applies equally well to any complex system, like systems administration.

FreeBSD jails allow an administrator to use a single operating system on a single physical machine, and then partition that machine into logical application entities that are no more than an IP address, a name, and the files absolutely necessary to run the application. I’ve been running the FreeBSD jails in a development environment for a while, and I’ve been very happy with them so far. You trade off some of the high-availability aspects that comes with the higher-priced VMware licenses, but what you get is a simple, reliable system. I’m impressed with Jails, and I hope that similar Linux projects, like Linux-Vserver really take off.

If you are just dipping your feet into virtualization, or if you have experience, and are looking into something different, it might just pay off to look into the FreeBSD jails. If you have experience running the jails in a production environment, I’d love to hear about your experiences. Drop me a line in the comments!

by an anonymous user on Jan. 03, 2011
Linux also has that. It is called Cgroups. It also has OpenVZ and LXC. No need to struggle with old and dusty BSDs.

0 Votes
by an anonymous user on Jan. 03, 2011
Cgroups is *not* the same thing as FreeBSD jails. A jail has a dedicated hostname and IP address.

FreeBSD is not an "old and dusty" operating system. In fact it is an excellent OS that is leaner and more consistent than its (slightly) younger cousin Linux. Linux is a nice kernel, with a bunch of random userland tools thrown on top. It's very difficult to deploy proprietary software on Linux because every distribution has different userland tools/versions, and a different filesystem layout. FreeBSD on the other hand has a known set of tools & libraries that are part of the base OS.

The only thing I would say that is "dusty" is the methods of installing new software packages - especially the ports collection.

0 Votes
by an anonymous user on Jan. 04, 2011
@ anonymous #2

It is very difficult to deploy proprietary software on BSD because every variant (or call it hostile fork) has its own incompatible kernel, libraries and tolls, and bunch of other random things thrown on top. GNU/Linux on the other hand has a known set of tools that are part of every distro.

:-)

Seriously, on GNU/Linux, you have same kernel and same user-land on all distros, only versions can differ. Only thing that is really different is package management system, but you can repackage everything. On BSDs (net, free, open, dragonfly) you have completely different kernel and completely different user-lands and libraries which completely diverged over the years. And of course, different port systems. It only the three letters (BSD) that are in common for all BSDs, they are completely different systems in every way. All incompatible.

I don't get how you can play 'fragmentation' card while BSDs are more fragmented. I also don't get how you advertise easier porting of proprietary software when number of proprietary apps on BSDs is probably single digit. And I pity those who are going to use compatibility layers on mission critical servers. Besides, why care about proprietary software when we have FLOSS everything?

You are comparing FreeBSD with "Linux", and missing the point. You compare BSDs with GNU/Linux. You need to compare FreeBSD with some distro. Like RHEL for example.

If you compare only containers functionality, GNU/Linux wins. It has Cgroups, OpenVZ, and real virtualization: KVM. With FreeBSD, you don't even have virtualization. Just jails.

0 Votes
by Andrew on Jan. 04, 2011
Process isolation like BSD Jails, OpenVZ and Linux V-Server is great for reducing overhead whilst creating isolation between processes. As mentioned it is a chroot on steriods.

All of the above however have the problem that swap memory isn't allocatable per guest, and that by running a single kernel, any vulnerability in that kernel is a vulnerability to all guests.

If one were to use Xen or KVM (or VMware) virtual machines can be more finely tuned than the process isolation types above, whilst incurring additional overhead.

For me: on old hardware that cannot to intel-VT or amd-VMX, I run OpenVZ (courtesy of Proxmox) and on the new hardware I run KVM. (other techs are faster than KVM at the moment, but I wish to contribute to its adoption and continued development)

Regards

A/.

0 Votes
by Shane on Jan. 04, 2011
@Andrew

You mentioned that other techs are faster than KVM. This comes as a surprise to me since my expereince is quite the contrary. Infact I have found KVM to be the fastest of all the Virtualization technologies mentioned above. The only downside to KVM is that the hardware (CPU) must support virtualization. Unlike Virtualbox one cannot run it on a single core system.

That said, in terms of performance VMWare sucks, OpenVZ is somewhere in the middle but is not as fancy as VMWare. In my experience and personal benchmarks KVM blows the rest out. It leaves VMWare so far in the dust that I cannot even see in from my rear view mirror.

0 Votes
by an anonymous user on Jan. 05, 2011
Why is it that when anyone mentions FreeBSD in a positive light or proposes it as a viable alternative to Linux, all the Linux zealots come out of the woodwork and rip on FreeBSD or any of the BSDs for that matter? Isn't selection and choice a good thing, especially in the open source world? It is completely hypocritical for Linux users to persecute and spread FUD about another open source OS. Linux advocates have been complaining for years about Microsoft and others spreading FUD about their beloved OS, and now that they have some market share and power they feel it's their right to do the same thing to another open source OS. Of course I'm generalizing, but it's a trend I've noticed on several well-known IT websites.

I have to say that I have tried almost a dozen Linux distros and have had problems with each one. I'm talking about serious issues, too, like filesystem corruption and not being able to boot after installing patches. In my two years of running FreeBSD I have had absolutely 0 crashes and 0 data loss.

Linux is a good OS, but from my experience FreeBSD is every bit as good.

0 Votes
by an anonymous user on Jan. 06, 2011
BSD advocates are ones that claim that choice is bad. They are quick to make a point that Linux has too much choice and to much distros, and that all that choice kills kittens of proprietary companies who can't port their kittens to Linux and chose BSD instead. When someone does a reality check and shows that BSDs are more fragmented and more incompatible, and that it is supported by _less_ proprietary software vendors than Linux, than, probably due to lack other arguments, some other BSD advocate comes and pulls off "zealotry" and "fanboyism" slur. As in "those who prove us wrong are all damn zealots and FUDsters".

This situations wouldn't happen if BSD folks stop perpetuating MSFT-conceived myths and FUD about some kind of Linux fragmentation, or incompatibility or too much choice, or what FUD they invent next. BSD advocates are really hypocritical to accuse Linux advocates of things they are doing regularly, and accusing Linux of flaws they are too blind to see in their bellowed OS.

0 Votes
by an anonymous user on Jan. 06, 2011
As far as some positive examples of using jails, check out the following link: http://www.playingwithwire.com/2009/06/virtual-failure-yippiemove-switch...

0 Votes
by an anonymous user on Jan. 06, 2011
One traditional weakness about jails compared to similar technologies (Solaris Containers) and other forms of virtualization was the lack of control over the resources used. There is a project under way that will bring this level of control to FreeBSD 9 which will be released this year:

http://www.freebsd.org/news/status/report-2010-07-2010-09.html#Resource-...

0 Votes
by hun on Jan. 06, 2011
People often miss the whole point of a solution or product. In majority of cases the individuals believe systems and emotions take precedence over logical reasoning, coupled with lack of real hands on information or understanding. Often of course there is the corporate incentive system involved too, where company X is pushing A,B,C vendors products as these will reward company X in various different ways.

In the past I had a fair amount of involvement with most Virtualization products like VMware, Xen, Solaris containers, FreeBSD jails etc. It is absolutely unnecessary to argue which is better or worse as each of these solutions have their own place and use and all of them are perfectly viable virtualization options for different environments and needs.

VMware and Xen is perfect for mixed environments or consolidation, Jails and Containers excel in service/application level partitioning and security. There are endless options with any of these technologies. I have seen solutions where VMware was combined with Jails and this allowed hardware independent migration options and flexibility.

0 Votes
by Euan Thoms on Aug. 17, 2011
I've used VirtualBox on Linux, zones (containers) on Solaris/OpenSolaris, and jails on FreeBSD.

Although VirtualBox is not as suitable for production servers, it behaves and is managed in much the same way as the commonly used virtualization technology in the Linux world.

Solaris zones are the most technically superior implementation (IMHO) of virtualization. They copied jails and added a bunch of other cool stuff.

However, for what I do, I still prefer FreeBSD jails. They're so simple to manage and almost instantaneous to create/start/stop. So efficient, simple, elegant, robust, yet flexible. FreeBSD + ZFS + Jails + Ports = server OS perfection! :-)

I am much more comfortable on FreeBSD than on any Linux distro or Solaris/OpenSolaris. Aside from SMP and Xen Hypervisor, FreeBSD as a base OS, is much more mature than Linux. Linux devs have been copying design elements from FreeBSD for a long time, they still are. Linux was an accident, like VHS vs Betamax. Just goes to show the power of momentum.

I still use all three OSes mentioned above. I like them all, but they have different strengths.

0 Votes
by Euan Thoms on Aug. 18, 2011
I've used VirtualBox on Linux, zones (containers) on Solaris/OpenSolaris, and jails on FreeBSD.

Although VirtualBox is not as suitable for production servers, it behaves and is managed in much the same way as the commonly used virtualization technology in the Linux world.

Solaris zones are the most technically superior implementation (IMHO) of virtualization. They copied jails and added a bunch of other cool stuff.

However, for what I do, I still prefer FreeBSD jails. They're so simple to manage and almost instantaneous to create/start/stop. So efficient, simple, elegant, robust, yet flexible. FreeBSD + ZFS + Jails + Ports = server OS perfection! :-)

I am much more comfortable on FreeBSD than on any Linux distro or Solaris/OpenSolaris. Aside from SMP and Xen Hypervisor, FreeBSD as a base OS, is much more mature than Linux. Linux devs have been copying design elements from FreeBSD for a long time, they still are. Linux was an accident, like VHS vs Betamax. Just goes to show the power of momentum.

I still use all three OSes mentioned above. I like them all, but they have different strengths.

0 Votes
by Mihamina Rakotomandimby on Aug. 24, 2011
About bottlenecks: Neither the RAM Nor the CPU are the bottleneck. It's the hard drive.

I have a 9 VM (multiple usages) host, they completely kill the hard drive I/O.

Reference:
http://ostatic.com/blog/considering-freebsd-jails