Conrey.org

Initially I was going to post this article as part of an install and configure series that I’m writing, but quickly realized that it’s going to be a probably going to be a reference article for other stuff as well. Hopefully, as I continue to learn and pick things up, this article will expand as well.

Why Nexenta is different

If you’re coming from the standard storage arena, it’s hard to put a bead on Nexenta. First off, it’s just software. But more than that, at it’s core is open source software. Think of it as a gobstopper with three layers. At it’s core it’s based off of illumos and ZFS. Added on top is NexentaOS, a ubuntu userland that makes software management easy. If you’re ever tried Ubuntu, you know that it’s pretty simple to install updates. The third and outermost layer to Nexenta are the additional modules that add additional enterprise functionality to the storage appliance.

What is illumos

The illumos project started as a binary compatible distribution of OpenSolaris, the comunity supported distribution of Solaris.  Years ago, for me anyway, it was the most cost effective way to learn Solaris.  The intent was to create a straight fork that would stay compatible with any possible future releases that Oracle makes to the community.  It turned out this was a pretty wise move.  The OpenSolaris project within Sun didn’t adjust well to the transition [1].  Now illumos stands at the center of a new OS community, with a SOLID enterprise class foundation.  Many of the folks that were contributing/creating to the solaris codebase continue to contribute to illumos, and it will continue to be an interesting project to watch.

What is ZFS

In short, ZFS is an enterprise grade file system. It was designed by Sun to be a modern filesystem to handle modern hardware. To understand why ZFS is important you also need to understand how modern harddrives work:

A modern hard disk devotes a large portion of its capacity to error detection data. Many errors occur during normal usage, but are corrected by the disk’s internal software, and thus are not visible to the host software. A tiny fraction of errors are not corrected. For example, a modern Enterprise SAS disk specification estimates this fraction to be one uncorrected error in every 1016 bits, or approximately one in every 1.2 PB.[19] A smaller fraction of errors are not even detected by the disk firmware or the host operating system. This is known as “silent corruption”. In a recent study, CERN found this issue to be problematic.

As hard drives get bigger, we’re all seeing data corruption become more of an issue. The changes of corruption, both noticed and silent, increase. In vmware envionments silent data corruption can ruin your day. I’m actually not sure how common it is, but in a service provider environment I worked at we had VMs that would not let us do anything with the vmdk. It seemed “stuck”. We couldn’t storage vmotion it, and it would continually give us grief. After troubleshooting for couple days, the only thing we could do was clone it and delete the original VM. That’s worked fine, but it could have been a lot worse. As storage envionments get bigger, silent data corruption will eventually happen in your environment unless you protect against it.

ZFS checksums every block that is written and automatically repairs blocks where corruption occurs. This provides end to end data integrity allowing you to be confident there won’t be a problem with the data when you need it.

Beyond this feature, ZFS allows for heterogeneous block and file replication, UNLIMITED snapshots and clones, compression, deduplication, non-disruptive volume increases and a host of other features. I’ll work through (and link to from here) these features in the future, but it’s important to get some other base functions explained.

What is the ARC

As legacy storage vendors will tell you; Cache is king.  The ability to cache data efficiently or “move blocks” or “other fancy caching name here” basically does one thing.  It moves data that is most accessed to a faster tier, or more responsive, layer.  ZFS leverages the ARC as it’s front line cache for most accessed blocks.  This means if your Nexenta appliance has 24G of RAM, it’s going to get 23 or so gigs to use to cache the most heavily used blocks.  As RAM is added, the ARC cache grows.  RAM being A LOT faster than SSD’s this proves invaluable for providing FAST response times and lower latency to the client devices.

What is the L2ARC

Below the ARC ZFS can leverage additional cache devices, in most configurations today this is SSD drives.  Cheaper than RAM, but still way faster than spinning tin.  This is the 2nd level of cache that Nexenta appliances use for caching “warm” data.  This hybrid storage model allows for Nexenta to use slow SLOOOOOWWWW disks for it’s main storage pool of the working size set (average amount of blocks that are constantly active) fit inside both the L2ARC and ARC cache.

What is the ZIL

The ZIL (ZFS intent Log) is the write cache.  Random R/W is the bane of every VMware administrators life.  Not only does Nexenta leverage a read cache, we can cache writes to more efficiently lay down blocks on the slow disks to better handle write workloads.

How does all of this tie together?

For our lab environment, if you can move the L2ARC and the ZIL to an SSD drive, you’ll already start being able to use some of the performance gains that Nexenta brings to the table. Even in a two drive configuration, if one of them is SSD, and there is plenty of Ram available, Nexenta can start helping increase performance through it’s caching technology.  But that’s not it, Nexenta brings to the table a host of other features.

1. Nexenta provides a software stack that can deliver NAS (NFS/CIFS) and/or SAN (iSCSI/FC)
   storage that allows for high performance allowing users to leverage current commodity technologies (ex SSD for performance).
2. It provides UNLIMITED snapshots and clones
3. It can be configured in a fully active/active HA configuration.
4. It provides block and file level replication that can be leveraged for easy DR configurations either in a syncronous or asynchronous configuration.
5. It’s vmware certified
6. It provides compession, dedupe and inline virus scanning. (not post process!)
7. metro clustering
8. global namespace
9. (this goes on and on)

 

 

 

 

[1] The opensolaris project wasn’t alone. Oracle’s purchase of Sun resulted in the forking of LibreOffice from OpenOffice.org, and MariaDB from MySQL. Bryan Cantrill’s talk at LISA  this year pretty much sums up oracle in a nutshell.