theChrisWalker.net

As my last post said, I’m using ZFS and love the way it works, but am having trouble with the performance of ZFS-FUSE. It’s not unusable, I just expected more.

So I thought, OpenSolaris. It’s the original implementation of the filesystem it must be great, so I export my zpools and boot from the LiveCD (another great reason to try it).

However it wasn’t going to be that easy… When I try to import my zpools in Solaris I get errors stating the vdev’s are corrupt! Something like:

status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.

Scared I boot back to linux to find they are all fine.

Back to Solaris and they still look corrupt. So I search and find this on the ZFS-FUSE Google Group.

Sun claims best practice is to use an entire disk when assigning disks to vdevs, rather than just partitions. So that’s what I did in linux. However the Linux implementation differs form the Solaris one and whole disks added in Solaris get a single partition first and that is added to the vdev.

Great! Now I have these zpools I can’t use anywhere else. To test I used a pool I set up as a mirror and detached a vdev (one disk). I then repartitioned the disk to have one big partition and also wiped the ZFS pool info from it (easier said than done! Ghost pools seem to be a problem!). Then I re-inserted the new partition into the the zpool and waited for resilver. Then I did the same with the other drive, detach from pool, partition, attach partition and resilver.

Bingo. It works in Solaris! Now I just have to find a spare 750Gb disk so I can do the same for my JBOD style zpool…

I have a fileserver at home. I have done for many years now. The first incarnation was based on Fedora Core 3 (which will tell you *exactly* how long ago that was…) and was very simple. I had an old Xbox HDD (8GB Seagate) and 2 shiny new 200GB Western Digiatl drives for storage.

This was also my first major experience with Linux as a whole, so I was trying to keep things simple. So I did a default type install, with LVM on the 2×200GB disks to give one logical 400GB partition.

All was fine and dandy until I wanted to upgrade a couple of years later and not knowing much about LVM, wiped all my data. Yes, ALL my data. I had some important stuff backed up, but a lot of non-critical but frustrating to lose data (many GBs of painstakingly ripped and encoded CDs and Audiobooks for example).

I was gutted, and vowed not to use LVM again (note my solution to my mistake and lack of knowledge - never use the software again! Well thought out!). So I rebuilt the server using CentOS 4.x ( I can’t remember…) and added another 250GB disk. This time I kept each disk seperate with it’s own filesystem.

This worked fine for ages until the disks were nearly full and I spent all my time moving data from one disk to another to gain a few GBs extra on another disk. Nightmare.

So I decided it was time to start again and did some research into the available filesystems people use for fileservers, how software RAID can be implemented and so on. In this time I came across ZFS and I couldn’t believe what it could do. It seemed to good to be true. I now know this to be false and it can’t do everything, but it can do an awful lot!

So how can I use ZFS? Well, if you use any Solaris variant, then you probably already are. Otherwise chances are slim that you’ve encountered it. I didn’t really want to go down the OpenSolaris route, but instead stick to Linux where I’m more comfortable. This poses issues, as the licencing behind ZFS from Sun doesn’t fit with GPL, so no kernel support for ZFS. Instead someone (I’d love to reference but I forget his name, google will surely help - http://google.com/search?q=ZFS+FUSE) has ported ZFS to FUSE which is great except for the massive memory usage and the poor performance.

Those actually sound like quite big issues! Well, the memory thing is not so bad as it seems. ZFS is not memory instense (well, actually it constantly uses about 256Mb so it’s not light) but uses available memory cache to store data in case it needs it again. A good idea, as the memory is otherwise just doing nothing! The Performance is pretty poor though. It’s enough for my network and to stream music, pictures and video over my network. But single file transfer never hits much more than 8Mb/s read or write.

Now I’m interested in trying some of the other OS options, as ZFS is easy to “export” from one OS and “import” into another. What are the options?

• Option 1: OpenSolaris – Seems like a good idea. The native platform, has a good AMP stack so should be fine for my needs. Have heard some stability issues, but not so much with ZFS, but the OS itself.
• Option 2: Nextenta – Solaris Kernel with GNU userland. Supposed to be a good happy medium, sounds too much like a compromise to me…
• Option 3: BTRFS – different route totally but a new filesystem designed not to mirror ZFS, but to take the best of everything it can. Some aspects very like ZFS. Very promising project, but too young.

Kinda only leaves OpenSolaris… I’ll let you know how I get on.